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In t roduct ion 



The idea of assessing the validity of the 
Carroll - Sapon Modern Language Aptitude Test was 
conceived by Professor Peter Strevens soon after 
his appointment as Director oi' the Language Centre 
of this University, It is not self-evident that a 
test designed for English-speaking university 
students in Americ2 and validated there is necess- 
arily valid in an English university situation. 
The differences between American English and our 
own variety are obvious, especially differences in 
pronunciation. Secondly the American university 
tends to take in a greater proportion of the popu- 
lation than in this country. Thirdly the education 
of the American pupil niay be vastly different from 
thai of his Hnglish counterpart. Different methods 
may have been used leading to a strengthening of 
different aptitudes. 

The Language Centre has a wide variety of 
courses available. Here is a list of the main ones: 

1 ) Preliminary Language Year (PLY ) 

This is a course in the School of Compara- 
tive Studies for students who intend to specialise 
in Government, Literature, Art, Sociology or 
Language (starting 1971), It is a one year intensive 
course in Portuguese, Russian, or Spanish, Students 
who learn Russian will later specialise in the 
Soviet Area, and those who study Spanish or Portu- 
guese will specialise in the Latin American Area, 

2) Common First Year (CFY) Russian 

This is a continuation of PLY, By the 
end of this, year students will be expected to be 
able to read Russian texts in the original. Students 
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who have a good A-level in Russian are not expected 
to undertake an intensive course, but are admitted 
directly into CFY. 

3 ) CFY Spanish and Portuguese "Conversion* * 
Cour se s 

Students who had earlier specialised in 
Spanish ^'convert" to Portuguese, and those who 
specialised in Portuguese "convert" to Spanish. As 
in the case of Russian, students with a good A-level 
pass arc admitted directly to CFY. 

4 ) CFY French 

Students who choose to follow a North 
American Opt'ion arc expected to study French in 
their first year. This is the largest single 
group of students, all of whom are required to 
have at least 0-level French. 

5) Social Studies French, Russian , 
Portuguese, Spanish . 

These arc "service" courses for other 
departments, normally of two years duration. Ko 
previous knowledge of the language is necessary. 

6) M>A. in Applied Linguistics 

This is a course for "in-service" teachers 
who wish tc improve their qualifications. 

There have been other courses taught by staff 
of the Language Centre, some of which no longer 
function (for example, Russian and German for 
Scientists) and some of which have been started 
much more recently, e.g. M.A. in Linguistics, and 
Elementary Italian. None of tlie students from 
these two categories arc covered in the main part 
of the investigation. The section by Mark Alford 
is concerned w i t h s c i en t i s t s learning Russian, but 
this only applies to Part V of the Test (Paired 
Associates) . 

The Test was perforned in a language labora- 
tory, so that conditions were as near as possible 
identical for all the students taking the Test. 
Timing was done by master tape, which had to be 
turned over at the halfway stage. The invigilator 
also had to stop the tape at the end of Part I 
CSumber Learning) for the students to transfer their 
answers to a form suitable for automatic marking. 
Thanks arc due to the rcmbcrs of staff of the 
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Language Cenire who helped with invigilation and 
ma rk i ng . 

In order to carry out the multiple regression, 
the program XUS5 was used. Mr. Ian Russell of the 
•Department of Mathematics adapted the program to 
our necd^ and helped enormously in interpreting 
the data. He also contrived to get more information 
from the program than had so far been achieved in 
this university, in that he used it to obtain factor 
loadings. I have included^ in the text those 
statistics which were more readily understandable 
to the layman. For those who are more "statistically 
minded" fuller details are included in the Appendix 
at the end of the Occasional Paper. 

It is a difficult task to measure aptitude and 
to isolate factors for an aptitude test, hence 
come of the terms used in Carroll's Factor Analysis 
are somewhat vague, e.g. "sound-symbol association 
ability", "associative memory". It is also difficult 
to know how aptitude develops when it is exercised 
by the right sort of study. If it does develop, 
then surely it becomes more difficult to discern 
innate aptitude and to discriminate between the 
aptitudes of pupils of different environmental back- 
grounds . 

Notwithstanding all the difficulties involved 
in the overall problem, we hope that this work adds 
to the body of knowledge and that people concerned 
with language testing will find it useful. 
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■'!ean Cccre 
• Iccian Gccre 

■'>3viate 
Variance 

Ctandard I>3viaticr. 
7^cer.rile 



Average score. 

Score ar tr.c centre ;x)ir.t on the 
rar*>: order of results '.(-here half of tliose 
bein;^ tested fall on each side of tlie 
line. 

The anount by v;hich each score 
exceeds cr falls short of the nean. 

The sun of the squares of the 
deviates . 

Obtained Ir/ taicinf, tiie square rcot 
cf th.e Variance. Used to shci; how well 
the test *s3^reads' the population LeLnr 
tested. 

A pictorial nethod of denDnstratin^; 
distrihuiiion. Used to sho:: ho\/ nany 
students obtained ha; reny narks in a 
particular test or subtest. 

The hi^Tiest poLnt in a histop?an. 
7:.o nost frequent value in a distributicn 
CL'jA'e or histograr . 

A netr£5d of dencnstratLx the level 
of perfcmance of an individ'-iai in 
relation to a control or 'ncm' p^^- 
TJius, if a student is on the 2Cth 
percentile, 10^. of the students in the 
•lorn^ /:pnoup :?ould have been above hln, and 
39'*^ lo:;cr. rjr. advcinta^e of this nethod 
is that indiyid'jals are net penalised for 
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Ircinj; Icr.i* in the rank order of their ci/n 
froup • 

Ccrrc'a-ion A ncthod of corparinj; one test v/ith 

" — .^tlicr ir. numerical tcnri;. A correlation 

of 0 rjsans that the tests are^neasurinr, 
different thir^^'.s. A correlation of 1 
\x>\2ld ncan that they nr^2 ncasurinp the 
sa-ae t:dncs. A rninus correlation vrould 
cast suspicion on the Lntemal validit:» 
of the tests bein,'^ oorrjared, 

'^ac^^^ One of the conponents beinn ncasurcd 

'"^ bv a test or sul;rtest. Tactors nay _ be 

isolated by statistical means and ^iven a 
•.ieirhtinc relative to other factors in a 
rarticular test* 

^'c-^-'^a.:!^ A-.>thinr '-^^^ discri-dnates bf^tijeen 

.c^^L^. or^c Student and another e.r;. sex, school, 

year, course of study, previous lan.^j^e 
study, o;<ar. narJ: etc. etc. In this 
investir;ation there are nineteen vai'^iables, 
T^iidi fall into different categories (see 
Lelcv:) . 

".ultirlc r-e.-r^^ssion A netliod of rroducinf a prediction 

^ — ' " ^ equation Lv ccnrarinc variables. I^ie 

rri-nar:/ ain of ::i/u is to rredict. The 
object' of this invostifation is to assess 

'.roll "?>JC lias orerated in the 'Jnivcrs- 
it-' of Zsseic. \le siiall do tiiis by seeing 
ha.; nucli tl:e individual of .'ILAT 

contribute to the prediction of course 
narl: and Oisnination rvark and hovj a 
prediction './ith :iUM ca:r^ares with 
Predictions -ic^ can nake usinc previous 
ian^^rc study on its own. 
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Description of HLAT and its uses at Essex 



P, T. Culhane 



What is MLAT ? 

During this description I shall refer to the 
Manual for the Test (No. 5 in the bibliography) and 
to the Factor Analysis produced by John B, Carr-oH 
in the Journal of General Psychology, 59, pp 3-19 
(No, 1 in the bibliography), 

"The MLAT has been designed chiefly to 
provide an indication of an individual's 
probable degree of success in learning a 
foreign language, but it is also useful in 
predicting success in learning to read write 
and translate a foreign language. It is 
applicable in connect ion with both 'modern ' 
spoken languages and ancient languages such 
as Lat in or Greek'* [5] . 

Each student is given an answer sheet, a Test 
Booklet and a Practice Exercise Sheet. The first 
two parts of the Test are on tape, and timing and 
instructions for ail five parts are also done by 
the tape. 

The test consists of five parts as follows. 
Part I Number Learning 

Candidates are taught the numbers 1,2,3,4,10, 
20,30,40,100,200,300,400 in a language unfamiliar 
to them. They are given an opportunity to practise 
the numbers and correct their mistakes. This 
provides a kind of short-term reinforcement. The 
numbers are then read tfut at random and the student 
has to write them down, when he reaches the end of 




t-- 'r'jnffvT tiic number? r r torn.': uhitl. r.jy Im- 
r-r. 1 : t J hv .-nac hjne or 1-y moa::< of n f|^o-j^atiy 
w » ii-t d TTtiirkiJig sheet w i t p. holo^. 

**This seems to measure one aspect of the 
nerr.nry component of foreign iarif.u3j!e aptitude 
and may also have nr. *. 'Auditory alertness' 
factor" ( 5 ] . 

Part II r ii o n e t i c S c r : r ; 

A series of sounds is read, which corresponds 
in groups of phonetic symbols printed on the answer 
Sficct . i:ach of four cheMCCS i i: five multiple- 
choice questions 15 read. At the second reading 
only one choice is read, and the student has to 
decide w!-ich one. The sounds in each group cf five 
questions are phonetically associated. 

"This appears to r-easure what we have 
called sound-symhol association ability , that 
is the ability to learn correspondences 
between speech sounds and orthographic symbols 
It IT. ay also measure a sort of memory for 
speech sounds, and it tends to correlate 
Iiighly with the ability to mimic speecli sound5 
and sound combination in foreign 1 anguages " t S ] 

Part III Spelling Clues 

This is a multiple-choice test in which the 
student i2 asked to decide which of five meanings 
corresponds to a word which has a disguised 
spe 1 1 iny . 

e,g. luv 

A , carry 

B. exist 

C. affection 

D, wash 
L . spy 

Ihis test is highly speeded and, according to 
the Manual, tests English vocabulary and the same 
kind of sound-synbol association as is measured in 
Part II. 
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Part IV 



Words in Se n t e nc e s 



Students are given t'^o scnter^ces. I"" the first 
sentence a word is underlined and In the secord 
sentence a number of alternatives are underlined. 
The student has to choose the alternative in the 
second sentence which has the sar.e funttion as the 
underlined word in the first sentence. 

e,g, London is the capital of England, 
lie liked to f i sh ing in Maine . 
A B CD E 

"This part is thought to measure sensitivity to 
grammatical structure" [5]. As in the case of Part 
III, the test is speeded, though not quite tc s-jch 
an ext ent . 

Part V Paired Associates 

Students learn vocabulary and are then expected 
to choose the correct meaning of a xoreign word 
from five alternatives. 

e.g. mcp 

A . in 

B . on 

C. that 

D. enter 

E. ^art 

The stimulus for each item is a printed one, 

"This part measures the rote memory 
aspect of the learning of foreign 1 anguage s*' [ 5] , 

What is described above is the long form of the 
test. The short form consists of Parts III, IV and 
V i.e. the parts which do not have aural stimulus. 



Factors in learr.ing a foreign language . 

Carroll lists the following seven factors in 
foreign language learning, in order of importance. 

1. Associative memory. 

2. Linguistic interest. 

3. Inductive language learning ability. 

■i . J <i .Tm a t i c ? i s e s i t i v i t y » 

1- . L- „ »u": - s y S"' 0 1 a s s c c i a t i o n . 
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6. Verbal knowledge. 

7, Speed of association, (Ij 

Carroll's isolation of factors and his weight- 
ings are based on a large nuaber of test variables 
ana not solely the parts of MLAT. The tests which 
becane part of the MLAT were a«ong those used by 
Carroll and the table below is an attempt to 
represent Carroll's factor loadings in a non- 
nunerical way, illustrating only the parts of MLAT. 



Parts c 


FactorG 
f \ 


1 


2 


3 


K 


5 


6 


7 


MbVT 


I 


















II 


0 


+ 








0 






III 


0 








0 


f 






I\' 








f 


















0 









* = '"ost i"rcrtant factor in subtest. 

+ = significant factor, 

1 = factor xvhich T'^ys sone r^t in test. 

= factor with little or no sipiificance. 



It aay be seen froa the above that the fourth 
factor i.e. 'gramaatical sensitivity* aay have 
aore iaportance in MLAT than it had in Carroll's 
overall analysis. Factor 3 appears to have less 
importance. The weightings nay» of course, be 
intent ional . 
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Uses of the test at Essex 

The prinary reason for the use of MLAT at 
Essex is to find out whether or not it is valid in 
an English university situation. Students who 
cose to us have usually studied a language at least 
to '0' level, but not always beyond. We wanted to 
knew how auch MLAT could add to a prediction of 
their success, and whether or not the test could 
reasonaifty be used as a selection cri terion . The 
pr iaary reason for the use of the test was therefore 
to invest igate the predict ive validity of the test 
itself. 

Other uses of the test aay be described as 
'fringe benefits*. At the beginning of the year 
we were able to circulate an early warning of 
students who were expected to be weak, so that a 
close watch could be kept on their progress during 
the first few weeks of an intensive course . MLAT 
has also been used, along with other factors, to 
determine which *strean' a student should be in in 
French courses. It has also been possible to 
advise students how they stand, according to MLAT, 
in relati09i t«9 other students. This is particularly 
useful in dealing with Mature students who have 
little or no foraal training in languages and who 
feel that t hey sight like to specialise in language 
study. 

We have not used MLAT as a selection criterion; 
it is the object of this investigation to decide 
how far we nay do this. 
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rhe Internal Validity of iILAT 



?. T. Culhane 



The operation perforned by a typical test or exanination is 
threefold: 1) It provides a ran); order i.e. it neasures tho 
level of ccrnretcnce of each stxident vMch places hin in relation 
to other students taking the test. 

2) It niay be used for r caned ial instruction, by 
irKlicatinc individual points of wealoicss. 

3) It'suGCests a pass mark, or mniinun level of 

acceptability. 

It has been sufx^sted that 1) ciboce is xwt »relevant* to 
students and that what should e:q^ect is that students should 
attain a self -satisfying level of ca:?«tence at \Jhich point an 
exanination becones * irrelevant* anyvjay. I do not \dsh to enter 
t^iis argunfint here. I do wish to poij\t out that the concept of 
•rarJ: order' is Lr^crtant in a test x*iich is intended to deteminc 
hovv '.^11 a student ray be e:^)ected to learn a foreign language. 

For the purTcses of this investigation ye have data fron the 
follc^/ing fytjups of students. 




Table 1 



Vear 




7> 


7^ 




Sr 


3o 




icrn 


n* V > 




(cri) 


(PLY) 










33 




20 


3 






76 


Conparative 




GO 




12 


1 


6 




01 


Studies 














36 






58 


8 


13 


1 


6 








37 






IG 




21 


76 


Social 










15 




31 


c6 


Studies 














76 




I" 


36 






5 










274 


10 


U5 


U7 


23 
































02 


125 




:kyte T = 


Frcnah 




cri : 


' Ccrncn 


first 


Year 










TLV • 


: ?rcliniivir\' i 




car 




r.ussian 















= ::^ranish 
Table 2 !.\roors of Students Schools. 



Voar i 


Ccnr^arativc 


Cccial 






Srudics 


Studies 






76 


76 




66 


01 


36 






05 


76 


172 




263 


233 





In cidditicr. tc the data frxn our ex. srader.ts we have also 
a liri.tcd orcunt of data from ATierican experrnents on ::LAr Csee 
[ 1 1 ► ( - 1 ( 5 n . ::e also have, throu^Ji the cooperaticr. cf 
:ir. Johr. S-Lith cf the Technolocical University' of Tiath, histcsrans, 
ncan scores and standard deviations on about five huxxired c::?scr'^- 
ations. I sh-all nake reference to these data in due course. 
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,le 3 :'e£r^ ItcT-darti Deviations for the test overall arc 







Ilin 


Max 


Standard 
Deviation 


Medlar 


full test ^^r:.T* 




M3 


179 


22»G2 


127. S3 


Short fcrr. CllI') 


£5.72 


19 


111 


15.11 




i: c:: 


2S tcl 


0 
3 


43 

30 


8.83 
3.40 


3S.17 

2c.ue 


III 


20.58 


3 


50 


6.23 


20.3c 




2S.56 


5 


41 


6. CI 


25. D£ 


(:-) 


17.57 


0 


24 


5.8C 


10. C3 



^ the :iean Scores that ?art III and IV arc 

-he Tcs^^^if"^-'--!' and ?art I is the easiest subtest. The larne 

j^r'^^-c^ the ^:can and the Median Scores Ln tests I ar.d \ 

^s'-^rcbab"'" a r«:1ecticn of the fact that naxirrsLm possible nar>:s 
^^^r^ c^z^ls.ot d larf;e nicrber of students in these subtests. 
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r.:'2:3c vJ^c i/-ra ^re^er.te'. lcs:icie these cf 
:'esr:.:e::* i:. i 3 ; , 

Percentile Ussck ra'.: /r-cricar. 'Ccllcrc rr^5rjr.cr.' m:v scores 





1 r •» 














; p 




* » ^ 


135 


TC 


137 




13^; 




132 








127 




119 


^0 


^ 22 


111 


110 


30 


115 






-»"^ 


1C7 


3C 


2- 




02 




?£ 


c 


ei 




7' 




501 




277 


tear. 


121;. u 


lis .'^ 


113. C 




13 




2C.6 



/zj ::c nifht expect the rriear score cf *rcur is 5ccc:>hat 
hi-'hcr thin that cf the Atverican sarrlc. Tr.e :>ec^cr. for this is 
rrcjal'ly that ti*:e /ncrican urJ-versitv s:.'Str". teres to recruit a 
\:L^cr section of the population ti^at curs ":c03 and that pccrlc 
'.itiC enter the Schools of tor^Tarativc arxi Social :--radie3 tend to 
be rcoplc '.;ith o^cricncc of lanf7Ja<:c ar-:: rr people -..'ith hi^h 
ap tir^e. The fact t-'uit the Ctandaix: Lcviaticn fcr our pro^jp is 
Icn.-cr indicates a banchin^; at the hiph.er end cf the scale. 



TalJc Porcanriles. 
the .rrrurc of *Ccllcfe 
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7t\esQ sr.c'j^z c-e studied in relation tc the idea of a 
'ncrral dLsrriyszLcr, cur--e*. In a norrvii distribution one vjould 
c:<pect a snail -jrl^ of people v;ith a lev; mark, a lar^e nur?l)er 
in t^.c rdddle rar.res and a snail number at the top of the scale. 
Tnus ±.0 c'-z^'c vcjIc lock somethLx lii^e this; 



lie. c: 
Crucentn 




Ir. the rJ^BZcrrars the distribution is represented in bloc}3 
of stu:lcr.rs cbtairij"^ rnar?:s ;/ithin a smll range. In the case 
of iLLsrc.-ra- A (sec ever) the Ic.^jst rork obtained by any student 
•.*as -3 and this iifjdent was the only one to appear in the ranne 
1*3 - 52. Zi\- people had a nark of ber.veen 53* ^ 62 etc. The fiode 
of the histCiT-ar. is centred on 137*5. 

I:isrcj;ra-ns 5 C 1 E i F (see over) 

'^.ly the histcrra-ns for tests III and IV shoi/ anythlnf; like 
a noirral dlstributicn curve. The Mean Scores in these tests are 
Icvjer in corr arisen -^"ith the other tests. In the case of Test II 
the MOX is net quite on the naxinun rark. This nicht lead to the 
speculation that if the test had been tried on a less able 
population the .'Sode vr^uld nove towards the centre. Ihe mean score 
at t-he Lfniversiry of 3ath is 110*3 i.e. 1U*1 laver thaB that at 
Il3sex. T^icre are reasons v^^*^ one nicht expect this to be the 
case. 3ath specialises in Science and Technolocy and its students 
arc less iLkely to ha%-e experience of Lancuages. Secondly, there 
is a tendenc:,* fcr st -: ?ent3 to be accepted Lnto Science Faculties 
wit-h la;er 'A' level lualifications than those ejq:«cted of 'Arts' 
srwidents. There ray an I.Q. difference. These reasons are 
sr^eculative and are net supported by definite evidence. Poy Cox 
[15) obseiv€iS that students in Corrvirative Studies at Essex are 
less •..'cll able to perfcm in reasoriinc tests than are Social 
Scientists and ?ir,'sical Scientists, riowever this nay be, there 
is no 5icr.ificant^ difference ber^^een the perfomance' in I^LAT of 
sr^dents m Social Studies and those ir. CorT^>arative Studies at 
Ilssex, and tr.e .near, score is si^^uficantly higher than the one 
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ciraLned by 3ath scientists. 

The 5atr. histcjira'ns are ver^' sinilar in shape to the Essex 
ones, and it is rrobaL)ly not a useful exercise to d i ^licate then 
here. Ihe. r^^c histocrams for Test II do not differ appreciably. 
A conparative lock at the mean scores of the subtests illustrates 
en interfistin^ point. 

Table 5 Ctaparison of Mean Scores in 3ath and Essex. 





Ilssex 


Bath 


Difference 




34.88 


34.0 


.88 




25*81 


24*4 


1*4 


III 


20*58 


16*0 


4*58 




25*56 


20*9 


4*7 




17*57 


15*0 


2*57 


Total (Full Test) 


124*42 


110*3 


14-13 


C3ricrr Fom) 


63*72 


51*9 


11*82 



3v far the potest difference in the nean scores occurs in 
Tests in end IV. Tests I, II and V account for only 4.85 (about 
S^'j of the total possible in these tests) of the difference of 
m-13. The tctai difference 14-13 is 7% of the total possible 
sccre in the Icn^ form of the test. The difference in the short 
forr. is 11-82 -.nich is 10% of the total possible in the short 
fom. The short fom on its own, therefore, gives a better 
discrinination between the tvxD groups of rtudents. 

Correlations 

Table 6 



Correlations 


illAT I 


riiAT II 


:iL/vr III 


:iL/vr IV 


lOLAT II 


0*425 








:I1AT III 


0*211 


0*295 






IHJC IV 


0-417 


0*418 


0*280 




:i/c V 


0*385 


0*351 


0*232 


0-375 
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rix^r. the aiiove it nay be seen triat Test III has the lowest 
ccrpelaticn f ifures • The reason for this ray be that this test 
i:r t:;e cnly one to be hi^-yily cpceded (Ivrr Shepherd - see below), 
Tnc fact of a lev; ccrralation may be a positive ratiier tlian a 
rx2f,ativc factor in relation to the test as it ircans that it is 
vicir^- 3cr>ethin^T wJaich is specific to itself. 

rvir.ciral Corrx^nents 



*ac^ 


:r^' T I 


:uy-.7 II 


:iLf\T III 


MLAT IV 




Variance"" 




>c:i 


C* 23U 




0. liSl 


0. 380 


85. 39 






(-0.029) 


a 875 


(-0. 092) 


(-0. 153) 


22. 11 




^rw 373 


(-0.1U3) 


(-0. 050) 


(-0. 120) 


0. 90G 


£. 39 




U79 


(+0* OIU) 


(-0* 152) 


0'e53 


(-0. 090) 


02 




(-0. 215) 




(-0. 037) 


(-0. luC) 


(+0. 0U4) 


1. 02 
117.93 



- victor^ 1 IS probably Ciirroll's Factor C or 'Associative .'lertDry'. 
Zz be a nistcuce to^atterrpt to ally any of the reraindcr' 

'.;:.th -Carroll's factors since each corresponds fairly closely vn.th 
a 3i--vle subtest, as follovs:- 

factor 2 :iIAT III English Vocabularv 

factor 3 :iI/.T V Rote Merrcry (Literal) 

factor u IV Oraraatical Sensitivit" 

factor S ML/YT II So\nd'Syrlx)l Association 

Ab>ilit^/, 

: The f ipjre in the variance colum measures how nuch variation can 
be acco'jnted for by factors e.g. 85/117.93 Cannrox. 72^i) of the 
var^tion ber-:f:in individual students can be accounted for b'^ 
factor 1, 
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"a^leS Differences between sexes and schools. 



Total riLAT 



Exnark 



+2-95 



+0-53% 
-0-50% 



+0-93% 



Corr>arstive Studies 



+1-15 



+0-33^ 



i;cne of the aix?ve figur>es are statistically sicnificant. 



Conclusions 



1. It would appear that the test overall is too easy for 
our students. 

2. Iherc appears to be a lack of balance in the test i.e. 
?arts III and IV are aCTreciably harder than the other 
parts and accxjunt for nore of the Internal variance. 

3 . Test III docs nor correlate highly with the other tests. 
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The Predictive Validity of MLAT 



P. T. Culhane 



All students who enter the School of Compara- 
tive Studies have some sort of Language qualifica- 
tion. This is not necessarily the case in Social 
Studies, but the total number of students in the 
sample without any knowledge of a foreign language 
{12] indicates that nearly all students who enter 
the School of Social Studies also have had some 
experience of a foreign language, even if this 
experience has been gained only on a casual basis. 
It ' normal under our system to use previous 
atta^iiiment in examinations as a predictor, and to 
admit students to university on the basis of their 
previous success in examinations. The quality of 
»0' level and 'A' level examinations as predic tors 
has (pften been questioned. In this University the 
School of Social Studies keeps a constant watch on 
the correlation between '0' and 'A» level results 
and university exams. The most recent observations 
(by Michael Parkin and Giles riomewood, 1969 mimeo- 
graph) seem to suggest a positive relationship 
between the grades of '0' level and final degree 
results in Social Studies. They found ([16] page 
10) that 'Overall performance at "0' level, 
measured in % good grades gained, is strikingly 
associated with Part II performance." Thus in 
their report quality and not range is found to have 
the most significant correlation. A further 
observation is that 'performance in Mathematics 
and Science 0-levels is favourably associated with 
Part II performance; this is not repeated with any 
other category of subject we measured.' Thus some 
predictor variables are more efficient than others, 
for the purpose of Social Studies. John Heywood 
(Lancaster) in a paper on student wastage [17] 
discusses the correlation between 'A' level grades 




and psychological tests with final performance 
irr. S - 12). He quotes an investigation at 
Sheffield by Austwick who found that French 'A* 
level correlates very hiphly with final degree 
perfornance in Fiench. This is likely to be true 
with other foreign languages as the skills demanded 
at 'A' level are still a necessary component in 
'finals* performance. A, Davies [18] finds that 
*GQ" (KQ. score and achievement in major school 
subjects) is a ?ood predictor of success in foreign 
languages, but his conclusions are very tentative, 
Ke have not tried to correlate MLAT with the 
reasoning tests used in the University, as we were 
not concerned with the possible use of MLAT as a 
rer lacement for 'A* levels. Our problen is a 
slightly different one. 

The scope of the investigation 

The aim of the investigation is to see how 
well MLAT works in our situalion, Ke already 
have a certain amount of information about each 
student who enters the Language Centre e.g. we know 
how pany 'O* and 'A* level subjects he has and we 
know his previous attainment in language. What we 
want to find out is how well MLAT will predict on 
its own andhowmuch it improves on predictions we 
nay make using only the information we have from 
the student's previous language study. It might 
be possible to work out a rank order of attainment 
by a 1 lotting points to 'A' level grades and adding 
tnese together for each student. Ke decided not 
to do this but to take the information in its 
crudest possi!>le form. We were left with the 
following categories: 

1, Number of'A'levels in language, n 



A 

Number of'O'levels in language, n 



0 



Number of Languages studied on a 
casual basis (including failed 
•0' levels) 



It was not the object of the exercise to see 
*.hcthcr previous language study made any difference 
to the score in MLAT, as we found that at this 
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sta^e we were involved in a 'circular' argument 
i.e. 'do people study language because they have 
high aptitude or do they have a high aptitude 
because they apply themselves to the study of 
language?' In our situation it was impossible to 
be conclusive, indeed an experi men t with a totally 
different type of population (e.g. at the Secondary 
School level) would have to be set up in order to 
investigate this problem. I quote the following 
Mean Scores for our popu lation by categories, 
merely out of interest and so that the work in 
producing them might not be wasted. 



Previous Language Study No. Mean 



Students with no language 12 S6* 17 

Students with only 

'casual' languagestudy 37 102-54 

Students with less than 

three languages 535 119-30 

Students with three 

languages or more 117 149-91 

Overall 501 124-42 



A further predictor which was taken into 
account is the Audiometry Test which was given to 
about 50% of the total population. This is a 
standard medical test which is used to measure 
hearingloss at different frequencies. The results 
are presented in the forr? of a graph for each ear. 
Jack Kay, of the L^npuaee Centre, who was in charge 
of the administering of the test, and I decided to 
classify the results subjectively according to an 
arbitrarily defined mean. We produced A, B, C and 
I) classifications, in an attempt to make the test 
measurable numerically. Since the audiometry test 
is essentially a method of detecting a physical 
deficiency or illness, it is not a test of aptitude, 
and it would he a mistake to try and evaluate it as 
such. If a person has significant hearing loss, it 
is necessary to take this into account in teaching 
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hin, in other v,crds, to take remedial' measures to 
prevent the deficiency having too great an effect 
on his studies. The test is design ed to point to 
deficiencies, not aptitudes, and should only be 
taken into account where deficiencies occur. 
Interesting wcrk on this topic has been done at 
the Fieginald M. Phillips Research Unit at the 
University of Sus5ex ( sec [19] and [20] ). 

For the purposes of evaluation we decided to 
take the overall MLAT mark and its component parts, 
he wished to fiDd out the following information:- 

a) Whether *ILAT as a whole is a cooU predictor*; 

b) IVhethcr it is better than previous language 
St udy . 

c) Whether the component parts of MLAT would 
contribute to a prediction of later attain- 
rent and by how much. 

d) Whether the prediction wc could produce is 
more aprlicable to some languages than to 
ot::ers » and again by how much. 

e) Whether there were any significant differ- 
ences between sexes and schools. 

The Criterion Variables are what we might expect 
MLAT to predict. In this case there are two 
criterion variables, course-mark and examination 
n:ark. These were available for about 95*6 of the 
population. The other S% are people who started 
courses and for some reason did not stay long 
enough to obtain any assessment, or people who were 
not assessed for any other reason. 

Bias Variables arc a) the different courses i.e. 
whether a s t ud e nt is first or second year 

and b) the different languages 

s t ud i ed , 

These arc two reservations which one must make at 
this stage. Those reflect to some extent upon the 
quality of the data. They are as follows:- 

\) The Coranon First Year French course is 
streamed, but since no information on this stream- 
ing was readily accessible the data has been 
analysed as if no such streaming had taken place. 
The result of this is to underestimate the predictive 
efficiency of MLAT. 
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2) Multiplicity of Languages. Given that much 
higher marks tend to be awarded in Russian, for 
example, the statistical analysis attributes this 
disparity entirely to a difference in marking 
standards. This is not to deny the possible super- 
iority of Russian students, but merely to accept 
the fact that such things may not be determined 
objectively. 

Cor re lat ions 





Multiple Correlations 


Canon ica 1 


Pr ed ic tors 


Course Mark 


Exam Mark 


Corre lat ion 


Bias ♦ PLS 


0-410 


0. 547 


0. 548 


Bias ♦ MLAT 


0* 388 


0.518 


0. 519 


Bias ♦ PLS 
♦ MLAT 


0-433 


1 

0.586 j 


0. 587 



It will be observed from the above that the 
multiple correlations are higher with Exam Marks 
than with Course marks. In order to assess whether 
MLAT is a better predictor than PLS we compare the 
figures on the top line with those on the second 
line. We would expect the figures on the third 
line to be higher since more predictors are 
operating. The crucial question is "BY HOW MUCH?" 
Thus it is obvious from the above, that 

a) Previous Language Study (PLS) is a 
better predictor of course marks and 
examination marks than is MLAT. 

b) There is an improvement when we add 
MLAT , but this improvement is not very 
significant. 

c) MLAT, when used in isolation.is quite a 
j^ood predictor of examination marks, but 
not of course marks. 

He have said (p 2 ] that the purpose of a 
multiple regression is to provide a prediction 
equation. Below are listed the prediction equat- 
ions for course narks and examination marks. The 
parts of MLAT are separated for this purpose. 
Where they do not occur in the equation they Jo 
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not cont rirute significantly to the prediction. 
Course Mar^ ( x ) 
Without ML^T 

X = -i-*?:^. ♦ 0. SSn^ - 1- 17n^ 48. 52 9.99 (if 
Russiar.) 1. 83 (if 1st year course) 
With MLM 

X ■ 3. 5Cr.. - 1. 45n- 0. 093m, 0. 107in, 
.-I L 13 

- C- IS-rr, - 0- -913a 4 3^ 95 ♦ 9* 77 (if 

4 

Russian) ♦ 2*11 (if 1st year course) 

Fror the above it is possible to deduce that 
course nrarxs in the Language Centre are on average 
well above the 40% pass mark.. It is also possible 
to see that not all the component parts of MLAT 
have relevance to the prediction of course mark. 
The number of languages studied on a casual basis 
(n) and the nark for the audiometry test (a) have 
a slightly negative contribution to the prediction. 
We have already decided to dis regard audiometry as 
a predictor. The figures above confirm that this 
is a right decision. The negative contribution of 
the nuntber of languages studied on a casual basis 
is probably caused by the fact that the figure 
includes failed '0' levels. 

, Let us take an example and work it out on the 
basis of the above prediction equations. 

A. Brown nas 2 'A* levels and 3 '0' levels in 
language. He decides to take Russian in the Prelim- 
inary Language Year and his MLAT score is 132 (36mj 

♦ 25m- + 30r, ♦ 21m. ♦ 20m^) 
2 J 4 5 

Mr. Brown's course mark ( in round figures) accord- 
ing to the prediction figures ought to be 

10 ♦ 3 ♦ 49 10 ♦ 2 72 
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If we include MLAT in the prediction we have the 
followinj; picture, 

7 ♦ 3-6 ♦ 3 ♦ 4 ♦ 43.95 * 10 ♦ 2 = 71.5 

The second prediction should be a better one than 
the first, but the difference between the two 
predictions is not very high. 

Exam marks (y) 

Without MLAT 

y = ^'^^n^ ♦ "^'^^n^ ♦ 1.68n^ ♦ 49 -41 - 4 -74 (if 
French) - 9-17 (if Spanish) 
With MLAT 

y - S-S^n^ ♦ 2.74nQ ♦ l-05n^ ♦ 0.268m, ♦ CllOm^ 

♦ 0-250n^ ♦ 0-156m^ ♦ 33-86 - 4.19 (if 

French) - 8-90 (if Spanish) 

We have already observed that both previous 
language study and MLAT are better predictors of 
examination marks than of course marks. This is 
reflected in the above two equations by the 
increase in the share of the prediction of *A» 
levels in the first one, and the increase in the 
number of parts of MLAT figuring in the second. 

Let us return to Mr. Brown, and see how he 
would have fared in the prediction. Again in round 
figures . 

Without MLAT 

16 ♦ 13 ♦ 49-41 = 78 

With MLAT 

12*5*9*3*5*3*34 - 71 

Again, the second prediction is lower and 
probably has greater accuracy than the first one. 

Of course it is not possible to say that the 
above predictions work in every case. Thev are 
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merely suggested by the figures we have obtained. 
Conclusions 

i.. MLAT is almost as good a predictor of 
likely success in foreign language 
assessment as are previous attainment 
tests in languages, when these are taken 
in their crudest form. 

2. The amount it is possible to add to a 
prediction of likely succ ess when one 
has in formation about previous language 
study, is insignificant. 

3. As we already have information about 
previous language study in the vast 
majority of cases it would be a waste of 
time to attempt to make this aptitude 
test a universal selection criterion for 
the Language Centre. 

4. j'he test could be useful in the following 
circumstances 

a) If a student wishes to find out 
whether or not he has a latent ability, 
which has not yet manifested itself in 
terms of attainment in language learning.* 

b) In doubt ful cases, for the 
'streaming* of students in a large group 
((^.g. French) although it is doubtful 
whether this could be effectively imple- 
flien t rd . 

*^ In view of the fact that we are not 
certain of whether or not results in 
MLAT have a significant dependence on 
previous language study, do we compare 
che results of a person in this category 
with the overall norm or with people in 
his cwn category? 
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Performance and errors in the MLAT of students of 



'proved' linguistic ability . 



r. Lee 



Int roduct ion 

In the second term of the M.A. Course in 
Applied Linguistics, eighteen of the twenty-three 
graduate students of the Language Centre, University 
of f;ssex, sat the full version of the Carroll and 
Sancn Modern Language Aptitude Test. (MLAT) 

The aim of this paper is to record and, where 
possible, evaluate the results of this test session. 

A) First, before looking at the individual sub- 
tests and scores, let us run through some of the 
basic assumptions implicit in the MLAT. 

The MLAT was designed "chiefly to provide an 
indication of an individual's probable degree of 
success in learning a foreign language*' [5:3j*, 
Its main use then may be seen to lie in its predict- 
ive or prognostic value. In analysing the results 
of the M.A, group, however, we are more concerned 
to show whether the MLAT scores reflect the measure 
of success one would probably expect of a popula- 
tion who, by comparison with a random population, 
are of proven linguistic ability. 

This type of test predicts how well a person 
is likely to learn a foreign language, and is used 
to select persons likely to profit from foreign 

• All references for quotations will be shown 
thus - that is, number 5 on bibliography, page 3. 
Other references will be by author and year only. 
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language instruction. [Mackey: 1965] 

Since the members of the M.A, group have 
already been whown (by examination) to have 
profited from foreign language instruction, we nay 
hope to show the validity for the MLAT. Its 
validity (i.e. how well it measures what it pur- 
ports to measure) would usually be calculated by 
establishing a correlation coefficient between 
test results and the degree of success or failure 
in studying, subsequently, a foreign language. 
Nevertheless, we should expect to find a strong 
positive correlation between 'prov.^n ability* in 
foreign language learning and the overall MLAT 
scores. There would perhaps be significant differ- 
ences between the F.L. group (those who have 
pursued advanced foreign language study - especially 
recently) and the English group (those who have 
had little or no contact with foreign languages - 
in any formal way or for an extended period - 
since leaving school). 

Further research and continued interest in 
the MLAT and similar tests is required since they 
fulfil an important role. The importance of 
language courses outside the school classroom is 
being recognized at last. More and more such 
courses are provided for people in the services, 
in industry and commerce. This necessarily means 
more adults are starting what are often expensive 
series of classes. They are expensive of time to 
the employers who release staff to attend Colleges 
of Further Education, expensive of staff in those 
colleges, expensive of equipment in the form of 
language laboratories and so on. Selection, to 
enable the most efficient use to be made of time, 
money and resources, is essential. On what basis 
are we to select? With highly heterogeneous groups 
it is not at all a simple matter to choose people 
who are likely to benefit from a foreign language 
course. with a group of »streaned» school children, 
where their I.Q. is known, their ability in other 
subjects and, more important, their degree of 
motivation and application are also known, the job 
may be somewhat easier. Even if all this inform- 
ation were available for an adult group, it is not 
at all clear that a safe prediction could be made 
about success in foreign language learning. The 
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'•IlAT rresumes to measure al-ilities (see below) 
which are specific to language learning and which 
are possessed to varying degrees over and above the 
'general intelligence* factor. 

It may be argued that all people will learn a 
lane. 1 age given enougii time and supervision. But 
this 'given' is vital. In any real - 1 i f e si t uat i cn , 
::ro is probably the nost crucial single element. 
1 he !1LAT does not attenpt to treasure wh ether a 
rcrson will learn .i foreign language, but Ho"w well 
lio is likely to learn in a given time. The measure- 
r.t'p.t then is relative, ratlier tlian absolute. 

The MLAT is useful in other ways too. It 
possesses a diagnostic function - that is, it can 
help to point out specific areas of difficulty in 
the language learning task, as revealed by depressed 
scores in certain subtest s. Further mo re,' it iray 
help to set up 'fast' and 'slow' groups within a 
largo class. This sort of infornation is of great 
use now that many language laboratories have a 
'grouping' facility, allowing different subsets 
wit!; in the class to work on different raterial and 
at their own speed. 

In the writer' s school, candidates were selected 
fcr fnrcign language classes, in which some dubious 

crnrciiy was agreed upon: Latin, Tcrman, Spanish 
[Trench Loing compu 3 s o r y 1 , on the basis of their 
first ycnr Ln^^lish rrarks and I.Q. tests. Since 

were tauglit by g r arrra r - 1 r a n s J a t a on methods 
■.'-.ch rax maJ^e appeal irore to general intelligence 
."^n-: rrrblom-sclving abilities, tliere ray have been 
?t;r::e j:;siificaticn in choosing prospective Latinists, 
etc. in this way . Subs ecuent results may well ha*. •: ~ ' 
• j i;*^ t i f i ed ' tlie reans. It is doubtful whether a 
^.v. : i-nsed on general abilities would be 50 
f t c. t i \ 0 f o r a u d i o - H n gu a 1 or other orally c r i e r t c d 

.r,-05 ot' study, ui:ereas the MLAT appears to be 
cqiiall} able to predict success in iiny tvpe of 
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c tests have shown (iiarding 195^) that an 
trycut of a foreign language is a good 
pivdictcr of subsequent success in that language. 
Ino VbAT, however, apart fror being applicab:e to 
."!:•. . -.-e'ln ,:r , y.'wvs the sn:.:Q re li able infer'-:!::--.-. 
•n .V hrur -^r '^.^ -i ? again?! n rntter of wee; '-r 
tni: 'tr'-rKf' i b nr t - ou r s c 05rcc^nIl^ 
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u lorrpr i.c - " " wasteful Of time, yet the MLAT 
■Soften at u;'-.rst reliable i„ these circumstances. 

Sapon C:?^3" underlineU the danger of relying 
---"-u-:le-solving ability when speaK- 
•"ng Sf ihn;-.;; = -^e;istics of I successful artif.c.al 
language for rrrjr-cstic texts. 

i-'-.- lc present complexities and 
re-=olut'c.-5 rf code structure as reflected 
ir'the -^-vel—nent of human language, rather 
■ ilian in tr.e construction of an abstract code 
probler.'- (p-fS) 

Carroll :lrt.:} states that: 

..c3-4iiTv in learning to speak and under- 
stand a'fsreign language is a fairly special- 
i-ed taler--. 'or a group of talents), relatively 
indeper.der.-- of those traits ordinarily 
includec under ■intelligence'." (p.89) 

fRelativelv recent developments in transform- 
ntlonal oere--ive grammar too. though for very 
d ffer nf r = a; support this vi ew .) The • spec la 1 - 

i-ed- taler^ are still intellectual abilities and 
J; some unk^cwr. degree trainable [4:89 f n . 2 As 
such the^are rejected by Pi.,sleur. Stockwell and 
Comrey. {.1962) . 

"It had been hoped that foreign language 
achieve.-er.t could be predicted on the basis of 
int ellectual factors, such as the ability to 
"discririrate sounds, to induce grammatical 
princicles. and so on. Instead, it appears 
froE these studies that the two biggest 
factors in such achievement are the very 
general ones of verbal I.Q. and mo t i va t ion" (p . 24 ) 

There would seem to be no obvious reasons why 
the MLAT scores should be invalid for the M.A. group 
a^ a iiole or for subgroups thereof. The reason is 
that the MLAT is largely independent of certain other 
varia b 1 es : 

(i) Language 

•^Fluctuations in the predictive validity 

of the ^!LAT do not seem to be consistently 

correlated with language or even type of 

language." [5:21] [cf. 2:158] . 
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(ii) Previous language training 

"All the evidence which has come in to 
date suggests that performance on the test 
itself is not much subject to improvement 
solely through taking foreign language courses." 
[2:157] 

Yet, / 

"Amount of previous language training 
and success in previous foreign language 
courses (important in our group - E.J. Lee) 
probably constitute predictors which may make 
some unique contribution to prediction over 
and above the MLAT" [5:21] 

Any help is likely to be of an indirect nature. 
That is, previous experience may give the student 
a better idea of his own learning strategy - more 
specifically, how to organize, for example, the 
learning of the Paired Associates in Part V. 

(iii) Language Method 

"There was no systematic fluctuation of 
validity dependent on teaching methodology" 
- [5:22] 

Finally, the MLAT has been validated for 
literate persons of native (or near-native) fluency 
in English. All members of the group, it was 
established by questionnaire, are native speakers 
of Eng 1 i sh . 

The full version of the test which, other 
things being equal, is reckoned to be more reliable, 
was administered to the group under nearly identical 
conditions in the language laboratory. One or two 
members who were unused to the laboratory conditions 
experienced a certain fatigue but there is no 
evidence to show that this affected the results. 

Since all persons wore headphones any advantage 
gained in tests I and II [see 5:4 fn,3] would be 
equal 1 y shared . 

Following Carroll (1962), we may define 
aptitude as: "the time which would be needed by 
individual 'i* to learn task 'j* to a specified 
criterion of learning, on the assumption that... 
the task is presented well enough for him to 
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understand the task in the light of his general 
intelligence." [4:122] (cf. 2:159] 

More particularly: 

•♦The MLAT reasures certain learned 
capabilities of the individual which arc 
apparently rrcrequisite to reasonably rapid 
success ir learning a foreign language," (5:21] 

That these are • learned* presumably implies 
that they are learned during Lj acquisition and 
arc applicable to 1.-,, 

What these capabilities are has been variously 
assessed. Cocparc with the following, those given 
by Mackey (1965) p. 326 and Pimsleur (1964). The 
latter hypothesises that auditory ability is the 
factor which accounts for differences in people's 
language learning ability which are not explainable 
by intelligence or interest. 

Carroll (195S) lists seven factors; (see pp.11- 

Factor A: Verbal knowledge . This comprises a know- 
I edge of the vocabulary and structure of L^, 
which is not unlike verbal intelligence. 

Factor B: Linguistic interest * One of the most 
inpor tint yet rost difficult to assess, this 
t^jctor is .1 specific motivation, interest or 
facility with respect to unusual (presumably 
•novel* rather than 'weird^ * J. Lee) ling- 
uistic nat er i al s . 

Factor C: Associative Memory . This includes 
itnmcd iate (rote) memory. 

Factor 0: Sound -Syrbol Assoc i at ion * This is the 
ability whicr. represents the extent to which 
the individual possesses a knowledge of sound- 
synbol correspondences or can learn a novel s 
set of such correspondences. 

Factor E: Induct ive Language Learning Ability . The 
ability to induce grammatical rules and 
properties of a language. 

Factor T: Grar.ratical Sensitivity or Syntactical 
F luenc That Vs , a !5t?n5i tivity to the 
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functions of words in senterces and a facility 
in producing coherent verbal naterials. 

Factor G: " Speed of Association ", This appears to 
have no significant role Tn foreign language 
learning, and will be ignored hereafter. 

Not all these factors a re of even weight of 
importance. Factors U» C, V. are of greater weight 
than factors A, U, F, 

flow far and which o{ the above are measured in 
the MLAT we shall see more precisely in the following 
sections, when we look at the different subtests one 
at a time. 

In the considerations of the various subtests, 
we have thought it useful to do an error analysis, 
in the hope that this might lead to constructive 
criticism of the design of the MLAT, and also to 
sone real measure of internal validation of the 
test - (see item Ig on P.T. Culhane's circular of 
22,11.68) - if not immediately, ther. at some future 
date when other data can be considered together with 
that provided herein. 



B) TCST I Number Learning . 

"This seems to measure one aspect pf the 
memory component of foreign language Aptutude, 
but the part also has a fairly large speciific 
variance, which one might guess to be ;? Special 
•auditory alertness' factor which would play 
a role in auditory comprehension of a foreign 
language," [5:3] 

Since this and Part V both measure iremory in 
some form, it is interesting to note from Fig.l 
that there is a very strong similarity between the 
shnpes of line I and V on the graph. Note the 
lower score on both tests for Nos. 8 and again 
for Na-s. 11- 12, 15 -16. 

Speed was no problem here; all rembers of the 
group completed the test and the marks were high. 
A mean score of 38.3 out of 43 (or 89. Ul meant 
that th:s test had the second highest near, score. 
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Error Analysis^ : 

TOTAL: to errors 

The number items to be learned were composed 
of hnglish sound elements. An analysis of the 
ans^^ers required, in the main, hundreds, tens and 
units to be recorded. With the HUNDREDS there were 
15 errors, with the TENS there were 33 errors, with 
the UNITS there were 23 errors. This would seero 
to suxecst that in being presented with an unfamiliar 
sequence XYZ, then attention and therefore retention 
is better in the case of X, Z - that is the first 
and last elements in each case. Y. the middle 
component, is less well perceived and/or recalled. 

With regard to the test design, the process of 
recorying the digits into the marking slots appears 
to rrcsent problems, 

5 people niscopicd 1 item 

1 person miscopied 5 items 

2 person niscopied 7 items 

Of the IS items miscopied. 13 were failures 
to r.ark rcro on the score sheets. The trouble 
arrears to arise since there is no zero mark in 
ti;e hun0.rcds column, but only in the tens and units 
colur.n. 

The case of zero occurs in another context too. 
Ten of the 69 crros were on items c., n, - that is, 
kith no 'hundreds* digit required in the answer. 
Failure to distinguish 32 from 302 (item n) costs 
the subject one mark - the units arc correct, the 
tens wrong (lose one mark) and the hundreds ignored, 
lut, to write 34 for 304 (item d for example) costs 
the'subiect 2 marks - one for the tens and another 
fcr the* hundreds. This would appear to be a strange 
ir.consi stcncy which night be removed. 

The errors for the various items break down as 
f ol lows : 

(a) 9 (b) 9 (c) 8 (d) 1 (c) 4 (f) 4 
(g) 5 (h) 6 (i) 4 (j) 0 (k) 3 (1) 1 (m) 7 
(n) 8 (o) n 

The high error rate at thi? bej^inning hay be 
due to hesitation in the early stages of this first 



test , 
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Of the 69 errors, 15 were niscopyings - see 
above. The remaining 54 were comprised thus: 

(i) 34 confusions (e.g. 200/100 or 0/40). 
(ii) 13 wrong colunn (i.e. 10 confused with 100 
or 4 with 40) 
(iii) 7 omitted (i.e. 242 written as 202), 



(i) 31/34 of the confusions were between 
numbers of the same power of 10 - i.e. 100/400; 
1/3. Only 3 were of the type 2/40 - that is a 
double confusion between the basic root number 2/4 
and the column units/tens. 

Further 26 were confusions between consecutive 
digits 1/2, 374", 2/3, whereas only 6^ were between 
1/3, 2/4 and only 2_ were between 1, 4. ^ 

This would indicate that the closer the digits 
were together the more readily were they confused. 

(ii) The 13 'wrong column' errors were well 
spread over the 4 digit root-words (1-4). 

2 wrot c 40 as 4 

3 wrote 10 as 1 

4 wrote 20 as 200 
4 wrote 30 as 300 

N.B, The tens column is at the bottom of the 
con f us ion . 

(iii) Since there was no number to be learned 
for zero, it is strange that 7 should have written 
in unnecessary zeros. 

There appears to be no pattern of any one 
subject consistently making errors with any one 
nunbe r or digit. 



C) TlfST II Phonetic Script . 

"This appears to measure what we have 
cal led sound -symbol association ability, that 
IS, the ability to learn correspondences 
between speech sounils and orthographic symbols . 
It may also measure a sort of memory for speech 
sounds, and it tends to correlate highly with 



Note 
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the al)ility to minic speech sounds anil sound 
combinations in foreign languages. [5:3] 

The three 'minics' of the group (subjective 
impression) each scored 100^», but then so did most 
people. Speed, once again, was no problem. every- 
one finished the test, 10 out of 18 scored 100*. 
On the face of it the rest was too easy. It was 
The best done of the five subtests, the mean score 
being 2i>.2/3n or 97.4*v. 

Can one argue th;it the test is too simple? 
Certainly one would have expected the scores to be 
high fron our group. First, the sounds were those 
of tnglisli and thus familiar. Second, the sounds 
were distinctive or phonenic; there wercno slight 
alloplionic variations to d i scr ini nat e . Third, every 
renber of our group lias some background in phonetics 
and at least a passing knowledge of the phonetic 
script, altliough the diacritics on 5, I, J, etc. 
are Anerican ratlier than British. Fourth, every 
test iter, was of a CVC arrangement, allowing maxi- 
iTium distinction t)etwcen sequcnts. Consonantal 
clusters nay have ee n h ard er . 

Urror Analvsis: 



14 I tens were wrong, of vhicli 15 were minimal 
pair errors and 1 a r.ultiple error. 

The iters wrong were: 

Once eacii: 6, 11, 12, 14, IS, 20, 24, 50. 
Three tines each(!): 26, 27. 

(ij Five people ivere wrong witli the 6/6 
distmctioiTI It scens unlikely that this caused a 
profiler in d i s c r i n i na t i on , and seers more probable 
t}iat t!)c errors were due to the fact that these 
wcTo the rosi 'exotic* of the symbols used: 

Syn^n o 1 s : 

Alpha! et ic : t ^ J c s ; ; j e 

J.xotic: I I ^ ^ d tya^eyaya^ 

luo norc sui^jccts hesitated on 6/t> before 
rottin^;: the correct answer by changing their minds. 
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(iii) Three errors were made on ay/ly, which I 
would presume to be a confusion of orthography and 
symbol* On hearing /ay/, the sight of 'ly* on the 
page may appear the right version by association 
with the sound of these letters in normal ortho- 
graphy. 

(iv) The remaining errors were a/e; j/3 
and e/ey. 

Note that the consonant /vowe 1 errors are approx- 
imately even:- 6:8 respectively. 

One which this writer had trouble in hearing 
was the difference between the American vowel /«/ 
* hag * and /a/ ' dog ' . Surprisingly, no errors were 
na3c in items containing this opposition, though 
there were two occurrences of hesitation - that is 
marking the wrong one first and then changing one's 
mind. 



U ) TLST III SpeUing Clues , 

"Scores on this part depend to some extent 
on the student's Hnglish vocabulary knowledge. 
This subtest also measures the same kind of 
sound-synbol association ability as measured 
by Part II, Phonetic Script, but to a lesser 
extent. It is highly speeded," [5:3] 

There is no evidence from our scores that the 
Lnglish group did any better on this test than the 
foreign language group. 

Parts II/III may be thought of as measuring 
differing aspects of the same ability, Part II to 
Jicar phonemic distinctions and Part III to produce 
phonemes. {cf, - 4:1151. With the small numbers 
available, the scores for these two tests do not 
seem to correlate to any extent. (See Fig, 1). 
Certain members of t}ie group felt the re ray be a 
negative corrc lat ion since the ' rigci;r ' of t he one- 
sound-one-symbol presentation of II may have caused 
trouble when coming to Part III. However this seems 
unlikely, since the 'spellings' in III are in fact 
quite 'regular*, once the pattern has been learned. 

The test certainly was speeded - (see details 
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below). This test was the worst done, the pean 
score being 27," out of 50 or 55.5*. 

Error Analysis ; 

There were 60 errors altogether • that is 60 
items marked wrongly. These are shown in Table 4, 
listed in order of overall rank order i.e. accord- 
ing to total rau scores). Most tn iei have been 
arranged in this same order to allow Ci oss -compari- 
son • 

There appears to be no connection between the 
number attempted and the number correctly done. 
The 'number done* is that number from the start to 
the last one attempted. 

A complete survey of the errors analysed in 
terms of the multiple choice items is given in 
Table 6, Certain of these - bracketed thus: [ ] 
on Table 6 - require comment here. 

Firstly t hough , i tem 13 ' f ra j 1 ' (f rag i 1 e) 
uould seem to be more easily recognized by an 
American than a British speaker. However, this 
item caused no error. 

One general comment would be that the method 
of answering, no doubt because it was a speeded 
test, seemed to be to mark the first possible 
correct multiple-choice item and move on, rather 
than reading all 5 possibilities and selecting one. 
This judgement is based on the observation that 
only 6 positive errors occur after the correct item 
(marked X ) . 



If all the multiple choice items are contribut- 
ing to the test equally as distractors (if not them- 
selves 
errors 
every case, 



tne test equally as distractors (if not ther 
the correct item) then one would expect the 
to be evenly spread, yot this is not so in 



CO Item 2. 'rgunent*. All 5 errors were on 
A choice - ' regiment ' . The closeness of the spelling 
has overriden no similarity in their meaning. 

(ii) Item 4. »nme'. 6 of the errors were on 
A choice, * sea-animal » . presumably because of the 
sound -assoc iat i on with * anemone » (?) 
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(iii) Item 8. 'pie' (pica). 4 people omitted 
this - Dorc than any other item. The meaning link 
with ^ appeal ' (choice D), is perhaps a littit remote. 

(iv) Item 12. 'ndkat'. 3 out of 5 chose C - 
a ' species of feline '. So doubt the last 3 letter 
of tne item areof significance! 

(v) Item 14. 'snscr'. 3 out of 5 chose C - 
' respond ' ■ One can only guess at a connection, sav 
with • sne c r ' ? 

(vi) Item 22. 'kurs'. 6 out of 6 chose b - 
a ' race-track ' - representing a confusion of 'course/ 
curse ' . 

(vii) Item 50. Only one person attempted this, 
and he got it wrong. The item 'pes' followed bv c 
choice A, ' vegetable ' is likely to read ' peas ' with 
a voiced /s/ finally. iiowever, the final /s7, 
consistently with other items /(cf. 6,22) was voice- 
less, — 



H) TLST IV Words in Sentences 

''This part is thought to measure scnsitiv- 
i ty tc grammatical structure, and may be expected 
to have particular relevance to the student's 
ability to handle the grammatical aspects of 
a foreign language. As yet, it is not Known 
how much scores on this part are a reflection 
of formal training in grammar; at any rate, no 
grammatical terminology is involved, so that 
the scores do not depend upon specific remory 
for grammatical terminology." (5:3] 

l\hcthcr scores on this test reflect formal 
training or intuitive knowledge is not known. With 
the amount of formal training done bv this group one 
night have expected the scores for this test to be 
higher. Most members of the group found it difficult 
to assess how well or badly they had scored on the 
test, yet most felt this was their worst score. In 
fact this was not the case for the majoritv of sub- 
jects, A mean score of 31.4 out of 45 or 60.o* 
rakes this test the fourth best done overall. 

Ahcn comparing it with lU (see tables 4 and 
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31 we note that tlierc were more errors made on this 
te<t (195 to 60) on fewer items (45 against 50) 
anJ yet only three people did not complete, or 
nearly complete Part IV. After test III, whichwas 
); ighly speeded, some degree of learning may have 
cn?ucJ, which prompted subjects to work far more 
q i c r. 1 y in Part IV - even at the expense of accur - 
a c >• . 

on FijTure 1, we see that the curve for Part IV 
cives the closest approxination to tlie overall- 
score curve. The sliort form of tlie ML AT includes 
?2rl IV, indeed perhaps this is the best single 
test to record aptitude for language learning. 

Error Analysis ; 

The complete breakdown of errors is given in 
Table 7. Once again, as in Part III, certain items 
call for immediate comment. The increase of errors 
towards tl\e, end nay be due to: 

(a) increasing difficulty of the items - though 
wc can find no reference to this, nor is it detect- 
able from a rescrutiny of tlie items; 

(b) fatigue of the subjects; 

(c) increasing speed of work before tlie end of 
r }*e t i Rc 1 i ni t ; 

(d) any combination of the above. 



Without counting those er ror s -by -omi ss i on 
listed in Table 7, there are still a great number 
of errors, not all of which are evenly spread over 
the remaining four incorrect items available for 
cacb. it en. 



(i) Item 9. That two people should omit this 
at an early stage in the test seened significant; 
the stimulus item 'SPKHUINn*, too. On investigation, 
items 9,2 0,32,33,42,43 also contain -ING stimuli 
and count among the highest error itei?s in the test. 
In each case there are confusions between Nouns, 
\djectives. Participles and Gerunds in -ING. 

IteiP No. 7 is, in this sense, exceptional, 

(ii) Item 11. Comments on item 9 show that 
general similarity of spelling, part of speech and 
so on seemed of more importance in determining which 
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item was chosen than function in the sentence. 

..Tk i^*^"" ^''"''■^ ^^^^ =""1 C*R) Grant 

are both proper nouns. 

P4.ii'H J^^'" '^K^i"' surface similarity occurs, 

Pi£C£ of N IS equated with dawn of N in spite of the 
Tin-that the first is 0bj.-N7-and the second is 
part of a proposed ADV.PIIR. 

^iV,, ^8=*^"' compare ' swollen ' (s) 
and ' hidden ' (wrong Res). — 

uhnl.^^K C''"j""ctions - badly done on the 

Whole. Compare items 1, 15, 34. 

(vi) Items 31/40. Pronoun is linked with pro- 



noun • 



are linLd 7" ^"^ quantifiers 

are linked erroneously - by 9 people (where N=18!). 

•a number"^f'°° '°k''' ^'"^ ' P"^' ^i"" 'many' and 
a number of may be synonymous. 

thi^liifaJ""" "^""^ demonstrate clearly 

the surface connection we are claimine which h^t 
been the greatest single factor 'in^'ca^'^irre^rSr 
Item 37, especially shows that choice 0 - the on v 
choice^containing -SELF was wrongly se lect:;%y°";o' 

expliiaJioi!'^'"' ""-"^^ °' "° •--P>'^" 

is ni^L ^-"u 'ii' °bject of the verb 'take' 

F) TEST V . Paired Asaoci.ir.. 

^^^^ measures the rote memory asnect 
of the learning of foreign languages '" fs"? 

The m^ai scoTj^^%' llVo'eM'" , ^ ^ ""'^'^ ' 

it the third .es^\l,L\Z^:l of the^JiJe*.' 

At the tine of taking the questionnaire answers. 



50 



44 



u- .r. were retcsted on this subtest to see . 
all subjects t,e recalled. Listing 

if any i^e.s could sti be r^_^^^^^ „un,bers 
subjects by their overai ^ _ ^.^ _ j. 

No.l scored 4; " • ' -fhis „ould seem to 

long-term mcnory. 

i^T-ifer was further interested to Know 

(a) 4 scored 100% (1st =) 

•(b) the remainder were 5th. 6th, 7th, 9th 
respectively. = 1^)- 

on such small numbers even, the evidence seems 
conclusive. 
rrror Analysis : 

The complete analysis is shown on Table 8. 
Three brief comments are required: 

(i^ In item 1, 4 people chose A in spite of 
CiJ ^" ^^^^ I f^T- Mn' included in the list 

there being no word for in incxu 

of associates to be Ic.irncd. 

(ii) Of the other errors, none seems to be of 
imncdiatc significance. 

(iii) So,ne of the more -obvious- distractors 

did not cause errors. For example: 

. I . No 14: • ja ?< yes' 

No. 12: -hon / on'; Tn 21- 'ketc ^ Vite' 

17- 'wcner ¥ never'; No. 21. 

only one person wrongly chose 'voun^' for -Jioni' . 
{No. 18) . 

G) A few additional comments are necessary. 

1 It is clear fron-. the lack °f . ho>nogene ity 

tLr'Ll'fnd^JHdentlv -asurable components of 
aptitude. Conpare Carroll (1059). 

-The separate parts of the MLAT are not 
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highly int ercorrc latcd , and it ray be assumed 
that they measure somewhat different abilities" 
(p. 159). 

and this can be seen to be a good thing, for: 

"one critical consideration in this 
connection is that if two parts are highly 
int ercorre lat ed , one or the other nay be 
failing to provide unique in f orira r, i on . " [5:17] 

2. Owing to the comparatively spall si.7.« of 

the group and overall homogeneity rner« is little 

evidence to suggest that language: aptitude changes 
with age: [cf. 5:23]. Also [2:iSV]: 

"It appears that (-he test measures func- 
tions which do not chvin^e jereatly from adoles- 
cence to adulthood.*' 

The overall rank-crder positions tabulated 
with the age-groups can be seen below. 

Table 9. 



Age 


Posi t i ons 


25-30 


1, 


5, 


14. 


16. 


30-35 


2, 


4, 


6, 


10, 11, i:, 15. 18. 


35-40 


7. 








40-4$ 


3, 


9, 


17. 




45-50 


8. 








50-55 


13. 









3. With regard to the relative scores of the 
two sexes, it has been noted [5:23] that in the 
upper ZT^de levels the girls score higher. The 
statenc Vhait there is too little data available 
on adu : ^ is echoed in the present case, only 
one of the 18 subjects being a woman » 

4. When dividing the M.A. Group into 'English 
on ly • and Fore ign language sub-groups , we find that 
once again no clear pattern emerges as to their 
relative success in scoring high. If anything, the 
results arc what one would predict - that is, four 
of the six English group are well down below half- 
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wa>' in the overall order- 



The two groups are as follows: 



Table 1^, 



Group 



Positions (overall) 



iing 1 i sh 



3. 6, 12, IS, 16, 18. 



Foreign Language 



1, 2, 4, 5, 7, 8, 9, 10, 11, 
13, 14, 17. 



5. It has been said [5:23] that the MLAT 
shows higher validities for predicting success in 
'intensive' courses, where the 'language bath' or 
raxinum exposure teclinique is likely to be followed 
and where motivation is higher at the start of the 
course and likely to be maintained throughout, owing 
to the relatively short duration of the course in 
que St i on . 

The MLAT itself, however, does not pretend to 
r.easure motivation itself» ^lie writer included on 
his questionnaire certain questions in an" attempt 
to obtain some rough guide to the suljjects' attitudes 
to the test from which it may have been possible to 
guess at tlie degree of motivation. 

From the results of the questionnaire (see 
paragraph H below) it would seem that even this 
rough guide does correlate with final raw-score 
on the overall test, 

6, Altliou;;h tlie scores at tlie top end are 
fairly close together, the overall range of marks 
is not as homogeneous as one may have thought. The 
variance does not seem to he attrihuta!)le to any one 
factor. The most interesting (and perhaps signif- 
icant) reflection of the scores is in the 'motiva- 
tion scale* (see below). 

Finally, a word about the questionnaire and 
t:,c data collected, (See also Appendix A, b) , 

The questionnaire was passed round approximately 




47 



ten weeks after the MLAT was given. 

(The numbers of the paragraphs correspond with 
the question'numbers of the questionnaire - See 
Appendix A.) 

1 N = 18; 17 men; 1 woman. 

3* All have English as mother tongue. 

2 4 These questions were to give information about 
' the language background of the subjects. The 

ranee of languages was considerable. No one 
had studied less than 3 foreign languages and 
no one more than 8. The levels attained ranged 
from 'casual* to honours degree standard xne 
languages studied are/were: Arabic: Cambodian: 
Chinese (Case-study): Fijian: ^^^"^^ = ^^^"^^'J i ^„ . 
Greek (Ancient): Greek (Modern): Irish: Italian, 
Latin: Malay: Polish: Russian: Spanisn: Urdu. 
(16 in all). All formal training in these 
languages had been by the grammar -translat ion 
method . 

6. Degree subjects: (with regard to overall 
pos i t i ons ) : 

Table 11. 



Sociology 


- 18 




English 


• 3, 


6, 12, IS, 16. 


English fi Foreign Language - 


- 9. 


13. 


Modern Languages 


' 1. 
14 


4, S, 7, 8, 10, 11, 
, 17. 


Classics 


- 2. 





5 Languages taught: English; English as L^/^^^^^^" 
language; French; German; Irish; Italian; Latin; 

Russian: ^ . * • t 

7,8. The ain was to establish a 'scale of motivation 

thus : 

From strongest to weakest: 8c: 8a: 8b: 8d. 
(with additional material from q.7 if any). 
9. Similar to 7/8 with the scale running from 9a-9e 
As was stated above, (Paragraph G) the replies 
to these questions do bear a relation to the 
score's achieved. 

•54 
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Table 12 



Sank Order 



7/8 



10 



6 



S 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 



c 
a 
c 



c 

b 
c 

h 
a 
h 
b 

a 

a 

b 



a 
a 
a 

c 
a 



a 

c 
c 
a 



defeatist* 
i> C+panic)' 

h 

b 

defeat ist * 
c 



expected 
to do well* 
b 
b 
a 

no reaction 

c 
c 

no reaction 
a 
a 
c 
a 
a 
a 
b 



* These were actual answers given. We thought it 
revealing to include them. it 

10. See Table 12 (above). 

He felt that people who 
a high grade might have 
that reason. Note that 
ly expected to do well, 

(10c) and were lower down the scale the 
you so- takes on quite another meaning. 
It is interesting to note that of the five 
nerbers of the MA. group who did not take the 
thov'w ' -ere absent and the other three said 
thc> Here not interested in th^eir .scores nor 
in the test - mainly because they were Tinglish' 
and the test was concerned with aptitude for 
torcxgn language learning. 

Ir people consciously used a learning stratepy 

of on'\ '''' «nenonics 

or one Jvind or another. 

7th, 9th did have some strat- 



had expected to reach 
done better just for 
the first three obvious- 
For those who answered 
I told 



11 



1st 



Those placed 
cg>-. 

Ihosc placed 13th. iSth made sojne attemt. 

i ;t;ateg>:! "° conscious effort to employ 
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The figures appear to be significant. 
To enable the subjects to recall runemonics used, 
they were shown the list of paired Hnglish- 
Kurdish words. Therefore, this necessarily had 
to follow q.l2 - the retest. (See Appendik B) . 



CONCLUSION 



Unfortunately, significant generalizations are 
not possible from such a snail number of results. 
It IS hoped that the results of this group will be 
useful in research with other M.A. or sinilar groups 
which nay show up special features about results at 
the top end of the scale. This information may be 
useful to University staff organizing language 
courses - especially for non - specia I i s t s 
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APPENDIX A 



Copy of Questionnaire 
Nane. 

Are you doing a case study? Russian/ Chinese/None. 
Mother tongue. Specify if not English, 
Foreign language(s) studied: 

Laneuace/^^"^^^ of , How long since^ j^^. 
Language/ g^udy ' last studied ' 

Language(s) taught, (including English) and for 
how long. 

Degree subject(s). 

Why did you take the MLAT? 

Kere you interested in (a) your own performance 
(b) the test itself (c) both (d) neither? (that is 
no special reason). 

How would you assess your work and attitude whilst 
doing the test (a) all-out effort (b) keen to do 
well (c) take what comes (d) not really trying 
(e) utter waste of time. 

With regard to your own score, were you (a) dis- 
appointed (b) pleasantly surprised (c) 'proved' 
to be right about your own attitude? 

Did yow use a learning strategy in Part V - the 
Paired Associates Test? Specify. 

(RETEST of Kurdish vocabulary) 

Can you remember any of your mnemonics, etc.? 
Any additional comments. 

•(a) bilingual/in the home (b) audio-visual (c) 'oral* 
method (d) direct method (no English at all) (e) 
Grammar-translation (f) casually/short visits to 
the country concerned (g) other (please specify). 
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1. Two strategies which failed were: 

(a) on the basis of previous sub-tests, patterns 
of a morphological kind were looked for in the Kurd- 
ish words to separate Kouns fron Verbs, etc* This 
wated precious time. 

(b) Links between Kurdish and Arabic (known to 
the Subject) were looked for - in vain. 

Zi Three other more helpful strategies were: 

(a) To do the ones learned and puess the rest. 

(b) Remember items by their position on the 
page relative to other items. 

(c) Find the short words - for example * j a * , 
'e* were the easiest to learn. 



3. Other mnemonics were as follows. We have divided 

them up into sub-groups which should be self-explan- 
atory. In each case, the words underlined are the 
Kurdish and Hnglish equivalents, the other words 
being the link word or words* 

A. Sound Associations 

j a - /dzci/ - /dei/ - day . 
* hui - whec!! - fall . ^ 

* ke t e - c anel (all i terat i on ) * 
• c home o - Chomsky - body . 

hon - shuddering noise - cold . 
y o n g - young (Iiawk) -hawk. 

1 ah - opcn-nouthcd sound - (open-mouthed) 

wolf . 

X020 - so-so - casual - easy . 
(n e n t e - dainty sound - 1 ady . 
( n e n t c - feminine sound - lady . 
(n e n t e - Italian sound - Italians - 1 ady . 
( nen t c - dolce far niente - Italian - lady . 
ngo; - a dark word to say. 
ni - 'me' touch me - touch . 
hui - 'high' - high fall - fall . 

B . The look of the word 

* t s c p - step - step inside - inside 
( lohong - holong - how long - question - ask . 
(lohong - long for - ask for - ask . 
£ - like a bowl on its side. 

N o t e . One French speaker had to fight off a 
negative transfer for 'iatc = sun' 
j at e - jatto -* (Fr.) bowl . 



cf 
cf 
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C. Mean ing 

roo - kangaroo - the bush - primitive - 

cave paintings - art . 
wener - Vienna - ' Kultur ' - book . 
kete - Katie (personification of) - camel 

U. Connect i on s witli other language s 

lohong - like a Malay word - the "Malay 

word" means to ask . 
X020 - like a Mexican Indian word. 
hui/ e/ mi / kete - like Malayo-Polynesian 
word s . 

Note the coincidences of similarity marked by 

* = identically used by more than one 
person, 
cf = similar. 
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Table 1 



RAW SCORhS - listed in rank order of totals and 
equivalent %i\es from 'College Freshmen' scale. 





1 












j 1 1 C 


Rank 


I 


n 


III 


IV 


V 


Total 


Col lege 


Order 












(raw) 


Freshmen 


I 


4 3 


30 


38 


36 


24 


171 


99 






30 


34 


42 


24 


170 


99 


3 


42 


30 


35 


38 


24 


169 


99 




40 


30 


2 8 


A f\ 

4 u 




162 


99 


5 


38 


30 


38 


32 


23 


161 


97 


6 


35 


30 


28 


38 


24 


155 


97 




36 


30 


38 


3 3 


16 


1 153 


95 


{« 


39 


29 


42 


28 




r 153 


95 . 


9 


|43 


30 


24 




24 


150 


95 


10 


j42 


30 


22 


3^ 


22 


149 


90 


1 1 


141 


2S 


27 


29 


20 


145 


85 


12 


|36 


29 


31 


34 


13 


143 


85 


13 


■40 


29 


26 


30 


12 


137 


75 


14 


|41 


30 


14 


29 


18 


132 


70 


15 


132 




24 


31 


13 


129 


65 


16 


131 


29 


26 


28 


4 


US 


50 


17 


i30 


29 


12 


24 


10 


105 


35 


18 


141 

I 


24 


13 


12 


6 


96 


20 


Totall 


• 43 


30 


50 


1 

45 


2 4 


192 


r 


Poss . 

f 














M e a n 


138.3 


29-2 


27 . 7 


, 31.4 


17.5 


144.3 




S.D. 


1 






1 




20.95; 

» \ 



Table 2 

MEAN SCORES OF SUBTbSTS ANL^ TliLIR ORPhR 



Part 


j I 


1 1 


III 


IV ! 

1 i 


V 


Mean Score 


[38-3 

1 


29' 2 


27-7 


31.4 


17.5 


Mean Score (as "iages) 


i 1 
89« 1 !97 >4 

1 ! 


55-5 


69-9 


7:« 1 












Order of Success - based 
on '«agc mean scores 


rr "n— 


5 


4 


3 
— 
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-Table 3 

RANGh ANU MtUIA.V SCORES FOR SUBTESTS AND TOTAL 





I 


II i III 

1 

i 


i ! 1 

IV ' V 

\ 

! 


Total 


Range 


43-30 
(13) 


30-24 M2-I2 

i 

(6) 1 (30) 


42-12 ;24- 4 

i 

(30) j (20) 


171-96 
(75) 


Median Score 


36.5 


27.0 i27.0 


27-0 jI4-0 


133.5 



Table 4 • Table 5 

SUBTEST III SUBTEST IV 



Rank » 
Order .Done 
Overall ! 

i ■ 


Sco re 


Krong 


1 


Rank 
Order 
Overal J 


Done 

I 


Score 


Wrong 


1 


i 39 j 


38 


1 




1 


45 


36 


9 




! 37 ! 


34 


3 




2 


45 


42 


i 3 


3 


37 ! 


35 


2 




3 


45 


38 


r 7 


4 


37 • 


28 


9 




4 


45 


40 


5 


5 


38 : 


38 


0 




5 


45 


32 


13 


6 


29 


28 


1 




6 


45 


38 


7 


7 


42 


38 


4 




7 


45 


33 


12 


8 


SO 


42 


8 




S 


45 


28 


17 


9 


25 


24 


1 




9 


45 


29 


16 


10 


24 


22 


2 




10 


45 


33 


12 


1 1 


37 


27 ■ 


10 




11 


43 


29 


14 


12 


36 


31 


5 




12 


45 


34 


11 


13 


29 1 


2 6 


3 




13 


4 3 


30 


13 


14 


IS ! 


14 


1 




14 


36 


29 


7 


15 


26 , 


24 


2 




15 


45 


31 


14 


lb 


28 : 


26 


2 




16 


43 


28 


15 


17 


15 ! 


12 


1 




17 


27 


24 


3 


18 


18 j 


13 


5 




18 


39 


12 1 


17 


TOTAL 


60 


1 


T 


OTAL 


■ ■■ ■ 1 


195 
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Table 6 



ERROR ANALYSIS OF ?!ULTIPLL CHOICh 
ANSWIiRS IN SUBTLST III. 



! Choices 
Uer.| ABC U' E 0 

1 


( 

Total 


11 


1 






1 


X 




1 


2 


- 


2 


[5] 








X 




5 


- 


3 


1 


X 










1 


- 


4 


It] 




X 






1 


7 


- 


5 




1 




X 






1 


- 


6 


X 












0 


- 


7 


1 


1 




X 




2 


4 


- 


8 






1 


X 


1 


[4] 


6 


- 


9 






X 








0 


- 


10 


1 








X 




1 


- 


11 






1 


X 






1 


- 


12 




X 








2 


5 


- 


15 






X 








0 


- 


14 




X 








2 


5 


1 


15 


1 




X 






2 


3 


1 


16 


1 


X 










1 


2 


17 




X 








1 


1 


2 


18 








X 




1 


1 


2 


19 


X 












0 


2 


20 








X 






0 


2 


21 




X 








0 












(6] 


X ' 


6 


2 


23 




X 


1 




T 




1 


4. 


24 








X 






0 


3 


25 






X 








0 


4 


26 






1 


X 






1 


5 


27 








X 


1 


1 


5 
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56 Table 6 Continued 



1 

No. 
I ten 


A 

1 


B 


Choices 
C U 


r 
c 


U 


Tot a ] 


u 


1 :8 


1 

-1 


I X 


1 — 


1 




0 


\ 6 


39 


X 


f 








0 


-1 

8 


30 




1 


1 ^ 




1 


1 


S 


31 




1 X 






1 


1 


8 


32 




i 


i ^ 




2 


2 


8 


33 






! 

X ; 






0 


8 


34 






X 


1 






0 


8 


35 








1 


X 


1 


1 


8 


36 








i X 
I — 






0 


9 


37 








X 






0 


13 


38 




X 


1 








1 


14 


39 






X 








0 


15 


40 










X 




0 


15 


41 




X 










0 


15 


42 








X 






0 


16 


43 






X 








0 


16 


44 


1 








X 




0 


16 


45 






t X 






0 


16 








— I — 




0 


16 


r-jT- 




X 




i 


0 


16 


1 48 








X 1 


0 


16 


49 




^■ 


! X 


i 

i 




16 


50 


[1 J 


' \ 


1 

1 


f 




17 



KhY: A,B,C,U,l: = Multiple-choice possibilities. 



No, It en 


= Number of item in test. 


0 


= lirror by omission 


U 


= Unattempted. No. of those not 




completing the test so far. 


X 


= The correct answer. 


1 ct c . 


= N' umber of people choosing this 


{ 1 


wrong item. 


= Mentioned in text as si ;jni f i cant . 
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fcKROK ANALYSIS OF MU LT 1 PLH -CIIU I Ct 



f 
i 


Iter 


A 

n 


6 


Cho i c 

c 


.es 

U H 0 


Total 


1 




1 


1 X- 

1 






1 


•7 






J 






X 




1 


3 


1 


1 


X 








1 


< 








X 






0 


5 








X 


1 




1 


^ 




1 


X 








1 


1 7 


X 












0 


S 


i 


1 


X 








1 








X 


1 




'f2J 


3 


10 


2 


X 




1 






3 


1 I 









[4] 


X 




4 


12 


[5J 




X 


2 




H 


[7] , 


I 


13 


1 ^ ! 


X 








" 2 ! 




14 


i 1 f 




1 


X 


! 






15 






.1 


X 




[6] j 




16 






[4] 




1 


s 1 




17 




1 




X 


1 


3 




IS 


! X 


1 






1 


2 


h 


19 


1 I 


2 


X 




1 


4 






2 i 


X 


1 






[31 


1 — 




1 


X 






2 


! 22 








X 




! 0 


l 2 3 


X 




1 




1 


2 








2 


X 




I 25 I 






2 




X ! 1 * 3 


1 26 1 


X 




[8] 


1 


9 
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Table 7 - Continued 



r 

No 

1 1 em 


Choices 
A B C D E t) 


1 

Total 


2 7 


X 


2 






1 




5 


o 




X 








1 


1 


2 9 




1 




X 


3 


2 


5 


3 0 


2 


i 


X 






1 


3 


3 1 


X 


[•1] 




1 


1 


3 


9 


3 2 


2 




X 








1 


F 31 


3 3 


X 






[8] 


I 


1 


1 0 


34 


X 






[4] 




1 


5 


35 




3 




X 


. 3 


1 


7 


36 


I 

X 


[9] 


1 


1 




1 


12 


37 




X 




[10] 






12 


3S 


1 


4 


[7] 


X 




2 


14 


3 9 


! 


3 


3 1 X 


1 


2 


9 


4 0 


1 

X 


[5] 




2 


5 


12 


4 1 


1 1 3 


' [6] 


X 


3 


13 


42 


[loij. 


X 




1 


3 


14 


45 


[Slj 1 




X 1 


3 


12 


44 


1 i [4] 


X 


! I 
1 


7 


13 


45 




[10] 


X ; 


6 


16 



KHV: as for Tabic 6 
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hUROR ANALYSIS OF MU LT 1 PLL -CliO IC H 
ANSWERS IN SUBTHST V 



1 

No. 

Item 


A 


L 

B 


Choices 
C b 


I: 


0 


i Total 

1 


1 


14] 


\ 


1 


2 




2 


9 






I 






X 


1 




3 


X 






1 


[3] 


1 


6 


4 




2 


X 


1 




2 


5 


5 


[3] 




I 

- 


X 


1 


2 


7 


6 


1 


' X 


2 


I 




1 


5 


7 


-> 


! X : i 










8 


1 


i 1 X 1 




1 


2 


y } 






1 1 


1 1 


X 


1 1 


3 


10 ! 


1 




1 i 


2 i 


X 


5 




1 


[3] ! 




X 


1 


2 ; 


7 






X ; 


1 i 


1 


[3] 


2 ' "7 




1 


7 


X ! 






2 ' 


5 







nrr 


i 


1 


1 


• i 2 




X 








1 \ 






[^] 


1 


1 ' 


X 


\ 

L 


1 ' 


6 


1 7 : 




X 




I 


1 


1 • 


3 


1 


2 1 


[5] 






\ ! 


5 


19 i 








X 


1 ' 




4 


20 ; 


3 


2 


2 , 


X 


1 ; 


1 


9 


21 ; 1 


1 


X 


[3] 






4 


2 2 


1 ! 




X 


-) 






23 '. 


2 








X ' 


I 




24 , 


A 




J i 


[3] 


2 


i 


6 



KEY: as for Table 6 
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An Item Analysis of Parts 111 and IV 



I , Shepherd 



Since Parts III and IV of the MLAT appeared to 
give a more nearly normal curve than the other parts 
of the test, it was thought worthwhile to carry out 
an item analysis of these parts to discover what 
were the relevant factors operating. This analysis 
is related to Lee's paper above in that some of the 
operations he used on a small sample are applied to 
the total number of test papers, and that his sample 
results are compared with the results of the full 
ana lysi s • 

The 594 papers (this total is hereafter 
referred to as N) were Jivided Into six equal groups 
on the basis of total test score. The answers of 
each group to each question an Parts III and IV 
were then analysed to give the following information:- 

1. The number of students who chose the correct 
alternative. 

2. The number who chose each of the incorrect 
alternatives. 

3. The number who omitted to answer the 
question but answered at least one later 
quest ion . 

4. The number by whom the question was 
unattempted; i.e. that question and all sub- 
sequent quest ions were unanswered. 

From this information the foil owing factors 
were isolated for each question:- 

a) Fac i 1 1 ty t - The percentage of correct 
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answers (hereafter referred to as C)« 

b) Uifficulty ;« The percentage of positive 
errors Chereafter referred to as I:), aiuJ the 
percentage of omissions (hereafter referred to 
as 0). 0 may also be related to speed. 

c ) Speed : - The percentage of unatccnptcd 
questions (hereafter referred to as U). 

d) Discrimination : - C of the two highest 
groups minus C of the two lowest groups 
expressed as a percentage. This figure was 
analysed to show the contributions of diffi- 
culty and speed to discrimination. 

c ) D i s t ract ion : • The percentage of E 
represented by each of the incorrect alter- 
native answers, especially the most frequent- 
ly chosen one - the "major distractor". 

These factors and their operation in each of 
the two sub-tests are taken up below and illustrated 
in Tables 1 - 4'and Figures 1 - 4. In addition 
questions shown by the analysis to be of particular 
interest are individually discussed. 

I. Sub-test HI 

a) F aci 1 i ty : - C fluctuates between 96\ and 
351 in the tirst fifteen questions, in most 
questions failing between 90^ and S0\. Thereafter 
C decreases steadily from 801 at question 16 to 4\ 
at question 38 and remains at this low level until 
the end. (See Fig. 1.) 

b) Difficulty ;- fc and 0 correlate with C 
for the first fitteen questions and then remain at 
a uniformly low level. (See Fip. 1.) Only fifteen 
of the fifty questions have a figure for t of over 
51, and of these figures only three exceed 25t, 

The equivalent figures for 0 are twelve and nil. 
(See Table 1.) 

c) Speed : * Sub-test III is highly speeded 
(5:3) and so there is a figure for U as early as 
question 10. It rises sharply from 2\ at Question 
13 to S]\ at question 33, and then more gradually 
to 95% at question 42. Thereafter it remains 
above 95V. 
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As B and 0 cease to correlate with C after 
question 15, U begins to correlate apd retains the 
correlation until the end of the sub«-test. (See 
Fip. I.) 

d) Discrimination : - This figure fluctuates 
between 2\ and 33\ in the first thirteen questions 
and then rises more or less steadily to Si% at 
questions 23 and 24. Thereafter it decreases 
steadily to S% at question 38 and then remains at 
a low love 1 . (See Fig . I . ) 

It is clear from the observations made under 
headings a), b) and c) above that sub-test III 
falls into two parts: before speed becomes the 
dominant factor, and after that point; the division 
comes at or about question ISr 

For the first fifteen questions there is some 
positive correlation between discrimination and fc 
and 0, indicating that discrimination is by diffi- 
culty. This is apparent when E and 0 are taken 
separately, and is even clearer when they are 
combined, particularly if questions 14 and 15 are 
discounted as already considerably affected by 
speed. (Sec Table 2 and Fi.g. 2, Nos, 3,4 and S.) 

There appears to be no correlation between 
discrimination and distraction. (See Fig. 2, No. 2.) 

Of the thirteen questions discriminated mainly 
he difficulty, only question 12 has a discrimination 
figure of over 30*., (a lower limit than many test 
constructors would be willing to accept). 

When the speed factor becomes dominant after 
question 13 there emerges a very clear correlation 
between discrimination and U. It is a non-mono- 
tonic correlation, however, so that questions with 
a low figure for U (early in the sub-test) and those 
with a high figure for U (late in the test) show 
poor overall d i sc r i ri nat i on , while questions with 
a figure for U between 20** and 7S*, (questions 10 - 
31} have pood d i scrim i nat ion figures - between ZA% 
and Sl%. (See Fig, 2, So. I.) 

(It is perhaps worth noting that questions 
;,hich do not appear to follow the curve in Fig. 2, 
No. 1 arc those where difficulty and speed both 
contribute significantly to discrimination. (See 
Table ^ . ) ] 
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The nature of the discriminat ion/unattcnptcU 
curve shows an essential feature of a speeded test. 
There must be a \arRc numher of questions to force 
speed, and so the later questions only discriminate 
among the best subjects, not overall. Conversely 
the early questions will eliminate the weaker 
subjects, but again will not discriminate over the 
whole range. Moreover, in a speeded test difficulty 
only operates decisively, as here, in the very 
early questions, before speed becomes dominant. 
Lven'then, it may be argued, it is speed which forces 
errors, and, more obviously, omissions - there are 
few omissions in a non-speeded test. 

The discrimination pattern of Sub-test 111 and 
its contribution to the discrimination pattern of 
the MLAT as a whole may be seen more clearly when 
the figures for C in the six groups are examined. 
General ly they are in descending order correspond- 
ing to the order of the groups on the total test 
score. Negative discrimination is not significant; 
only thirty-nine out cf two hundred and fifty 
group discrimination figures are negative, and 
twenty-four of these are in the last fifteen 
questions where the figures for C arc too small to 
be relevant. In any case t:*^- average negative 
discrimination in all these cases is less than 2%, 

In the early Questions the figures for C in 
the six groups descend by fairly even steps, with 
a slightly snaller difference between the two 
middle groups. As the effect of speed makes itself 
feft, the differencebetween groups 5 and»6 bectf^^es 
prominent for a short time, and then the difference 
between groups 1 and 2, with a smaller increase in 
the difference between the two middle groups. As 
the figure for C decreases the difference between 
groups 1 and 2 becomes increasingly dominant until 
by the end of the Sub-test it represents almost the 
whole of what little discrimination there is. 

e) Distract ion : - As stated above (p. 63), 
onl y f i fteen ai the fifty questions in Sub -test III 
have a figure for H of over S*, and it seems 
reasonable to confine consideration of distraction 
to these questions. 

As Lee noted Cp.40)» the iru 1 1 i p 1 e -cho i c e 
alternatives do not distract equally. Of the sixty 
distractors in the questions considered, two are 
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90 . 100% of E, two arc 80 - 89% of F. , four arc 
70 - 79% of I, two arc 60 - 69^ of two arc 50 - 
S9^ of I, three arc 40 - A9% of E, two arc 50 - 59*- 
of h, five arc 20 - 29* of E, twelve are 10 - 19* 
of I, and twenty-six are under 10-* of E, (Sec 
Table 1 . ) 

Lee's comment that the speed factor resulted 
in a preponderance of errors occurring Jeforc tlic 
correct item (p40) is borne out by the full analysis, 
but the effect is less marked. 68*^ of errors occur 
before the correct item in the full analysis, com- 
pared with 90^^ in the sample. The significance of 
the effect is in any case reduced by thr fact that 
there are 112 choices before the correct item con- 
pared with 88 after it. 

f) Individual Que st i ons : - l.ee (pp. 40,41 ) 
picks out for comment seven quest ions having a 
stronp major distractor or a high number of 
orissions. It would secir appropriate to begin with 
these seven, commenting on Lee's observations in 
tho light of the full analysis, and then to 
exar. ine other questions, wiiich did not appear 
significant in the saniplc analysis but do appear so 
ir tile full analysis. 

1 . Qucs t i on 2 : r gunent . 

Ill c two analyses agree or the ri i o r d : s t r i c t o r , 
r c i r c r: t ; in the full a n n 1 y s i : T iz ccwr.x ^ for 92 \ 

I whcro r i 2i^% of Thi? is x'r.o '::p:ic5t 

m.'i i o r distractor and the t h i r .1 h i :: r < t f : g r t far 
1. in Sub -test III. 

2 . Question 4 : n mo . 

The two analyses agree fT. tr.o rn/r. r distract or, 
s ca - an i n:i 1 ; in ihe f u 1 ] annlysi*: ir ac^^'.r.t? for < T» 
at I. iv h e r c f. is 5 *J of N . In t h e f '.; « 1 analysis, 
}*. w 0 V C r , t ij e r 0 : s ^mother d : s i r a c t o r , " \. r ■• e s i: , 
accounting for o" I. Tl;is (Question r.as the 

ii I 5 h e.» ^> t f ; r: u r c for ! S u b - 1 1- s t III. 



\ I) I ' i i r 1 1 :, .1 i > s : .'^ t ). i ■ * ; r\ :i n s the 
h ; »; h c s t : : ;. u r c K ; • , ? t : \ - , c r : 1 i \r. :•. 1 ^- •"• : ; t 
i.as onl> ih' t : i. . sgKeM f::"urc *''r ' ' , and also 
!: h a I v u i' i u r t" : ■ r i . , 'i r. d f r r d ; ? c r : r i r. a t ; c r» . 
In t ;. e full a n a I ■ ■ i •> t li c rr. ,i ; r ,1 : : * r c - r ; s t r o 
! f roT J r r 1 o ? } . 
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4. Question 12 : n d^kj it . 

The two analyse- agree on the major distractor 
species of feline : but in the total analysis ^t 
accounts for only 4U of fc where E is il% of N. 
This IS clearly a puzzling question, however, as 
the figure for 0 is 25^, the highest in Scrb-tcst 
III. [The figure for the omission component of 
discrimination is also the highest in Sub-test III 
(See Table 2 )] The difficulty lies, perhaps, in' 
the number of steps that have to be nadc to reach 
the required word. The letter-name 'n' has Co be 
used instead of its sound, but it ha:: to be chang.>d 
to /in/; a syllable and a vowel, /i / or /s/, have 
to be inserted between 'd' and 'k' with no clue 
given; and finally 'a' has to be given its letter- 
name sound /e»/, which it would never have in suc^ 
a position in English spelling. 

5. Question 14 : snser. 

The two analyses agree on the major distractor 
respond; in the full analysis it accounts for 77% ' 
of E, where E is 36% of .V. This is the second 
highest figure for E in Sub-test III, and this 
r'^^f'^'S^^f^^",^^^ second highest figure for 0 

(^4% of N}. It is, therefore, the most difficult 
question in Sub-test III, but not a good one, as 
the discrimination is only XB%, of which difficulty 
only accounts for so that it is clear that both 

good and weak subjects were confused by the questi 
Like question 12 it requires vowel changes, letter- 
name sounds, and the breaking of English spelling 
rules to arrive at the correct word. The choice of 
^^^PP"^ IS perhaps explained by the reading of 
snser as anser (answer) as the pressure of speed is 
beginning to be felt. 

6, Question 22 : kurs . 

The two analyses agree on the major distractor 
race-track, but as speed is an important factor by ' 
this point, the figures for E and 0 in the full 
analysis are very low (9% and 2% of N respectively^ 
As Lee says, the error represents a confusion of 
^°"rse and curse . The confusion is perhaps 
increased by the fact that in the only other 
question in which 'u' occurs (question 2, rguont) 
It has the sound /ju/, and the general inc onsist - 
ency of vowel sound/symbol values has precluded 
the reading closest to orthography. 
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Question SO 



pes . 



The two analyses agree on the major distractor, 
vcget ab 1 e ; although in both, of course, very few 
subjects reached the last question. 

8 . Question 7 : K warn . 

This has the fourth highest figure for E and 
the third highest for 0 in Sub-test III, 18% and 
19% respectively. There are two almost equal 
major distractors (41% and 38% of H) , citrus frui t 
(yam?) and dwc 1 1 i ng (wigwam?) but the comparat i ve ly 
even distribution of errors indicates a general 
difficulty rather than the effect of a strong 
distractor. This difficulty possibly stems from a 
combination of the comparative rarity of the 
lexical item qua Im , and the problem of assigning 
a value to 'a'. This is the first time 'a' occurs 
in the Sub-test, but from the previous occurrence 
of other vowels, subjects would tend to hesitate 
between the letter-name value, /ei/, and the 
English spelling rule value /^/\ the value /a/ 
would not readily suggest itself. (In fact, apart 
from this question, 'a* always has the value /ei/ 
or /ao/, except when followed by 'r'.) 

9. Question 10 : thnkf 1 . 

This has low figures for E and 0 but an 
extremely high rrajor distractor, t hough t fu 1 {91% 
of E). The subjects choosing this alternative 
presumably take the letter-name of 'n* and change 
it to /in/ (as required in question 14) instead of 
TO /^n/ , thus manufacturing a (supposedly American- 
English?) word thinkful . 

10. Question 11 : kn f rns . 

This has fairly high figures for E and 0 (16% 
and 11% respectively) and a high major distractor, 
k i nd of tree (conifers?) (86%). It also has the 
sec ond h ighc s t figure for discrimination by 
omission (23%). (See Table 2.) Merc there is no 
clue to the vowel needed between *k' and 'n', but 
those choosing the major distractor were not led 
astray by this; they chose the right vowel but 
ignored x hv second *n* to arrive at con i fer s . 
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11. Question 28 : kataklizm . 

This has a high E figure for so late in the 
Sub-test (10\) and a high major distractor (88\ of 
E) , chemical reagent (catalyst?). katak lizm seems 
close to a **siinpl i f ied spelling" version of 
ca tac lysni y so it was perhaps the un f ami 1 i ar i t y of 
this lexlca 1 item which led subjects to choose an 
incorrect alternative occurring after the correct 
one in the list of multiple-choice answers. 



II. Sub-test IV 

a) Fac i 1 i ty ; - C decreases from 96% at 
question l to 10\ at question 45, vith individual 
fluctuations. (See Fig. 3.) 

b) Dijfficulty :- 0 is generally not signifi- 
cant in the Sub- test , as would be expected in a 
test which is not highly speeded; the highest 
figure for 0 is t% of As in the sample analysis 
(Lee p. 41), E is generally high. All except two 
questions have a figure for E of over 10\, and of 
these thirty-two are over 2S%; eleven exceed 5GU 

The general decrease and the fluctuations of 
C correlate negatively with E except for the last 
few questions, where speed plays some part in the 
answer pattern, (See Fig, 3.) 

The increase of E was noted in the sample 
analysis (Lee p. 42 ) but only as occurring at the 
end of the Sub-test. No evidence appears in tlie 
sample analysis of an actual increase in item 
difficulty, but as in the full analysis the increase 
of fc is apparent from the beginning of the Sub-test, 
it would seen that there is in fact a grading of 
item difficulty, 

c ) Speed ; - Sub-test IV is not highly speeded, 
but the appearance of a figure for U at question 22, 
rising to 37% at question 45 shows that speed is an 
increasingly important factor in the second half of 
the test. The figure for U exceeds C for the last 
four questions, although it never exceeds E, 

The effect of this is apparent in the distortion 
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of the correlation pattern of E and C after question 
57. (See Fig. 3. ) 

d) Discrimination ; - This figure increases 
gradually from S\ at question I to 45% at question 
34, with fluctuations. Thereafter it decreases to 
10%, again with fluctuations. (See Fig. 3.) 

Of the 45 questions in the Sub-test, only 
eleven liave a discrimination figure of over 30%. 
This is a small number for a Sub-test which is not 
highly speeded, and where, therefore, all questions 
would be expected to contribute to discrimination. 

There seems to be some positive correlation 
between discrimination and difficulty for the first 
thirty-four questions. (See Fig. 4, No. 3.) Kith 
the appearance of a figure for U at question 22, 
there is also a strong positive correlation between 
the U component of discrimination and U; both are 
rising - oidly. (See Fig. 4, No. 5.) 

After question 34 the positive correlation 
between the U component of discrimination and U 
continues, but is negatively accelerated. (See 
Fig. 4, No. 5.) At this point also, .the K com- 
ponent of discrimination falls sharply (with no 
clear correlation with E) and for the last few 
questions is a minus quantity. This counteracts 
the effect of the U component, leaving the 
correlation of the total discrimination figure with 
U a negative one for these questions, (see Fig. 4, 
No. 4) and establishing a non-monotonic correlation 
pattern for the Sub-test as a whole. (See Fig. 4, 
No. K) 



There is no correlation between discrimination 
and distraction. (Sec Fig. 4, No. 2.) 

As with Sub-test III, an examination of the 
figures for C in the six groups shovs the contribut- 
ion of Sub-test IV to the overall discrimination 
pattern of the MLAT. As in Sub-test III these 
figures are generally in descending order, although 
in this case negative discrimination is slightly 
more significant. Thirty-three out of two hundred 
and twenty-five group discrimination figures are 
negative, the average of these figures being 4%. 
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In contrast with Sub -test 111, the relative 
differences between the figures for C in the groups 
do not show any pattern of change through the Sub- 
test, although there are considerable individual 
f 1 uctuations . 

e ) Ui stract ion : - As all questions except 
one have a figure of E of over S%, the distraction 
pattern is clearer than in Sub-test III, 

Apain, the alternatives do not distract 
equally; of the 176 distractors in the forty-four 
eligible questions, one is 90-100% of E, seven are 
80-89'ii, eight are 70-79%, eight are 60-69%, twelve 
are SO-59%, eight are 40-49%, twelve are 30-39%, 
eighteen are 20-29%, thirty-six are 10-19%, and 
sixty-six are under 10% of E. (See Table 3.) 

In this Sub-test there is a slight preponder- 
ance of errors occurring before the correct answer 
(53%) where the number of choices is almost 
equally divided (91 before the correct answer and 
89 after it). 

f ) Individual Questions ; - As in the 
corresponding section of the examination of Sub- 
test III, the questions selected for comnent in the 
sample analysis will be discussed f i rs t* f o 1 1 owed by 
questions suggested by the full analysis. 

1 . Question 9 . 

It is seen as significant in the sample 
analysis that two out of eighteen subjects omitted 
a question comparatively early in the test. In the 
fu3 1 analysis, only eight subjects omitted it. The 
other questions with - i ng stimuli do not seem 
particularly significant in the full analysis, 
except for question 43. (See below.) 

2. Question 11 : They named him BILL 

Because of his military success during 

A (2i) 

the Civil War , the people made Grant 

B(l%) C (1%) D (96% of E) 

president of the United States. 

E (X) 

E is 29% of N. The two annlyses agree on the 



78 



72 



major distractor. 



As Lee says (p. 43) the general similarity 
between two proper nouns distracts attention from 
the difference of function between them. 



3. Question 12 : The company owns every 
substantial P I ECE of property in the town. 

Before the dawn of hi story , men were 
A(71%) B(l%) 

raising corn very much like what we grow todav. 

cTxT dT2T%) ITTTt) 

E is 37% of N. The two analyses agree on the 
major distractor. Again, as Lee says, surface 
similarity overrides functional difference. 

4. Question 16 : My finger became SWOLLEN 
from the infection. 



The child grew st rong from the heal ing 
A (X) B 



suns h i ne . 



The hi gh wall was nearly hidden from view 
CT2T) " U(85%) 

by the f o 1 iage . 

E (2i) 



E is 22% of N*. The two analyses agree on the 
major distractor. Perhaps there is more justifica- 
tion for the error as there is frequently ambiguity 
of function with words with -en (or -ed ) endings. 

5. Question 26 : Do A£ I say 

A 1 1 h ough the weather report predicted 
A(X) 

clear skies for today, i_t^ rained all day. 

B f6^o) CC60'i) D(13%) E(21%) 

h is 41)*'» of N. This question has the highest 
figure for 0, 6%. The two analyses agree on a high 
figure for h . Two of the other conjunction items 
mentioned in the sample analysis do not appear 
particularly significant in the full analysis, but 
question 54 Joes. (See below.) 
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6 , Quest ions 51 and 4 0 : 



31. Which one do YOU think it is" 

That one may belong to me. 
aTxI B ( 5 1 } 

Pleas e pay m£ before i,oing on vour trip, 

40. Which colour do YOU like best? 

This one suits me better than the other. 
AtX) B{63%) CJaTJ 

It makes no difference to me , 

U [S%) T(2S%) 

I is 57 -o of N in question 51, and S0% of N in 
question 40. 



The two analyses apree on the distracting effect 
of similarity of parts of speech. In the full 
analysis the effect of the major distractor and the 
significant minor distractors depends on pronoun 
being matched with nronoun, regardless of function. 

7. Question 56 : A NUMBbR of people applied 
f or t he pos i t i on . 

L ^^"^ many candidates who cannot offer 
A(X) B(7 7%1 C(16"i) DTT'O 

■ ore than t wo ye ar s * expe r i enc e . 

I: is 67% of N, the highest figure in the Sub- 
test, As Lee says, meaning plays a part, overriding 
functional considerations, and making many the maior 
distractor in both analyses. ^ 

S . truest i ons 37 and 43 : 



57. His wife bought liBRShLF a new hat. 

Why won't you tell me more about 
A(2%) B(X) C(6%) 

yourse 1 f than you did vesterdav. 
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B is 52% of N, 

45. The child hurt MIMSELF > 

Although I myse If would do that by myse 1 f , 
A"(6%) B (6%) 

Marv gained hers e 1 f the help of some of her 
C (83%) D(X) 

classmates . 
. E (5%) 

h is 53% of N. Again the surface similarity 
overrides the functional difference, making h er s e 1 f 
the major distractor in both analyses. 

9. Questions 38 and 44 : 

38. WHAT is this? 

I do not know wh at book vou want, 
aT4T) 

To whom do these belong? 
B(19%) 

Wh ich fellow is your brother? 

C ( 7 5 % ) 

Those arc mine . 

u(X) eTsT) 

h is 54%ofN, 

44. There is no POINT in going ahead. 

When the light changed, he stopped the 
A(17i) 

car . 

B ( 4 4 % ) 

A river flows down to the sea , 
C (X) D(3n%) BC8%) 

b is 46^ of N. 

Tlu- two analyses have the same major distract- 
ors. In Question 58, which is perhaps seen as the 
most likely niatch, as it, like what , precedes i£ 
in a question, and in fact could occur alone without 
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changing the meaning. 

In question 44 there is apparently difficulty 
in equating the post-verbal noun in a there Is 
sentence with a pro-verbal subject noun, river and 
so the post-verbal noun car or the post-verl»al 
adverb down are chosen, 

(Question 41 : We plan to take IT today. 

On the chance that he would see us, 

A (2^^) fi(44^) 

we took steps to put up n beacon. 
C(2%) \)iS2%) E(X) 

f: is 45^ of N . The two analyses afjree on the 
major distractor. As Lee says, the aajor distract- 
or matches i_t^ with s t eps by their relationship to 
take , ignoring the di ff erent status of take in the 
two sentences. The minor distractor links it with 
us , presumably because they are both pronounT, and 
both objects of verbs. 



Ihe large number of questions with a high 
figure for t means that a large nuinber of questions 
could be commented on, but a representative sample 
of difficult questions with stron" major distr act- 
ors can be obtained by considering those questions 
with major dis tractors accounting for over 70% of 
I:, where I; is over 2S% of N. This sample provides 
.Jlustrations of the various difficulties encountered 
by the subjects and the typical errors made , Of 
the thi^ v^n questions satisfying these criteria, 
six wc: .scussed in the sample analysis and have 
thereto: iready been commented on; the other seven 
arenow considered, 

11. Question 14 : SEVERAL were absent from 
the meeting. 

In spite of the manv proposals wh ic h 
A(4%) bT73 % ) CTTTT) 

were made, only one could be adopted, 
U(irOE(X) 

E is 51% of N*. This question is comparable 
with question 5b in that similarity of meaning of 
quantifiers overrides difference of function. 
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12. Question 24 ; lie drove FROM Boston to 
New York . 

To bje safe, he decided to^ buy spare 
aTI0%)BC5%) C(77\) 

parts for anv emergency. 

517) eTs%) 

H is 26^ of N. Here a preposition, from 
following a verb is matched with an inf in i t ive 
marker, to following a verb. 

13. Question 25 : He nailed the board TIGHT 
against the house. 

He a 1 ways did the job well . 

A{4%) bT2%1TT87%) 

He poured the pai 1 f ul I 

irr7T)ETx) 

E is 34% of N. The "adjective of result", 
t i gb t is patched with the adverb well. Both are 
Tn the same position, following the object of a 
t ran s i t ive verb . 

14. Question 2 8 ; The weekly meeting, usually 
held on Friday night, is a fixed ACT IV ITY 
of the Scout program. 

Washington was the first president of the 

A (8%) ' B (X) 

United States ; he refused the crown that sone 
C ( -n ) D(74%) 

of his admirers wanted h im to have. 

E is 35% of N. The noun head of the subject 
complement following is is equated with the object 
of ref used . 

15. Question 30 ; NONE was more curious to 
solve the riddle than I. 

gg\g rnment * s first task was to check 
A(7:: O hTPET C(X) 

the prescript ions written bv the doctors . 

UJtV) I:(12\) 
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E is 62% of N. None, subject of was is natched 
with governgent ' s , one of the modifiers in ^ noun 
phrase"^ sub j ec t of was . This is the second highest 
figure for H in the Sub-test and the major distract- 
or there fore represents a far greater percentage 
of N than C does, which seeiss strange in vie« of the 
apparently obvious di f f e renc e bet ween none and 
governnent ' s . It is possible that government is 
seen as the "si tuational" subject of the sent ence 
and therefore matched with none . 

16. Question 34 : 1 will buy a car WHEN I get 
the money. 

A ft er you left 1 as t night, raost of the 

A(X) cTTst) 

students reinr.i.ned until the end , 

E is 34% of N. When , a conjunction linkinj? 
main clause with a subordinate clause following 
is matched with unt i 1 , a preposition introducing . 
modifying phrase. The similarity of position and 
meaning has overridden the functional difference. 

17. Quest i on 43 : Some people onjov EATING 
c lams on tlTe half- shell. 

Hac kin_g his way through the t efcrain g 
A(82%) Bt?i> 

jungle, he found abundan t c v idence of the 

vani shed civilisation. 
E{5%) 

E is 54% Df N. Two ing forms arc equated. 
Both have objects buX their func: ion is quite 
di f f erent . 



Lee concluded (p.43)that the most fiencral 
cause of error was the tendency for subjects to 
pay attention to general and surface similarities 
Tiicher than functiorial ones in the particular sent- 
ences. The full analysis, illustrated in the 
questions discuss ed above, confirms this. By far 
the most common form of this error was the matching 



r r .xTXs 0^ speech regarJiess of fi:nctii)ri, l^ut there 
-ere rather forns, such as natchin.c by phonological, 
V. r - o 1 o f: : c a I or it:o r p h o 1 o i c.i ! s i n i i a r : t i e s , a n »i 
r n : c -i 1 n.c Vy pr s i t i on in re 1 at i on to . : -le r i t or-s . 
.•:f:crcnces between the stinulus sentence ana the 
ri'spor. 5e sentences in order of sentence elements, 
sr. J the occurrence of two-clause sentences obvioiislv 
r 2 :i 5 c >^ c i f f i c u i t y . 



c n c J u s i o n 



Koliowing Lee's pilot study, the full analysis 
■ -s indicated the factors at work ir. a speed and 
a r.cn-speeaed Sub-test of the MlAT and the relation- 
5-. :ps i-etween then. it has also shewn the pattern 
c: errors in each Sub -test as a whole, and specific 
difficulties in particular questions. It i 5 hoped 
t r;a t the i n format ion ga i ned from mis s t udy , part- 
icularly that related to discrimination, ray !»e of 
,cip to future test-constructors in designing 
iuitable aptitude tests for v>c at university 
cntrancelcvelinhritain. 
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Tilc MoJcrn Language Artitudc Test Part V and the 
Lcarninr of Forcicn Vocabulary 



M. "1 . A I ford 



The Scientific Lanpuafc Project at the 
University cf Lssex developed a new method of' 
learninr to read foreign languages in which the 
nre.T.o r i za t i o n o: vocabulary played a r r edon i nn n t ]^art 
Students icarnt about 600 high-freouency words 
drawn fror t;ic target literature. The resulting 
hirh t ex t -cove rare of knovvn words facilitated the 
learning of new uords hy deduction and ensured the 
maxinun refresr.ncrt of those already menorized. 

Grarnatical instruction was carried out using 
vocahu I a ry w:i i ch had prev i ous ly l cen 1 earn t . Th i s 
rerovcd lexical distraction and nadc the exnrrplcs 
nucn easier to renerjhcr. Since the examples were 
corrosec nf i::gh-fre(iucncy words, 5uJ)seauent read- 
inr; r^rovideii constant rerrinders of all the points 
w li i c h had be CTi t ud i e-d . 

The ML.vr Part V (Paired Associates) was used 
in designing the neRoriiation procedures. It v.as 
necessary that people of all abilities should be 
able to raster then and the test was used to pick 
suitable <;i:ijccts to undergo t he . expcr i nen t s . 

\ o c 3 i' u 1 n r i L.^ s of Cno wo rci s were n ernor i 7 cd by 
afjout Sj' inaiv:duals with sccre-s ranging fror. i> 
to 2 . K i \- c u : ; t c t -S who volunteered to continue 
rcrori:ation aCv-iuired over l,i)(>0 words and all 
could iiave crntir.ued. Tiicir 'll.AT scores ranged 
fror. 1 u to 2 . *i h r e e c f t ii c m did Russian, one 
?p a n i s ii and vr. c -J : J : h hun g .m r i a n anci Uuran i a n , 

*. 1 li r;-' r: v" : r i ;.it i on was carrtcd out tc a 

precise w r i 1 r : » r. . 1 r. 1 e a r n i n r i< u s s i a f . - ! ri g 1 i s h 
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paired associates (PAs), 28 were presented on 
cards in each session. Time spent structuring the 
background of each PA was kept to less than 50o and 
students were advised to introduce attcmpteil 
remembering as early as possible. Ihis was 
performed with thirds of the pack shufflirip at the 
end of each imperfect test. Finally all the cards 
were shuffled together and the process repeated 
until the first correct recognition of all cards. 

This procedure was designed to produce a 
certain amount of underlearning on a 2 - 3 month 
retention. The PAs which made up this underlearning 
were ascertained by tests at interference intervals 
of 140 and 270 PAs and the requisite rclearning 
carried out . 



The initial session represented by far the 
greater part of the memorizing effort and its smooth 
functioning was essential to the «^uccess of the 
method. The MLAT V was given to every student 
before the course began. Instructions for the 
procedures most suitable to each ability could then 
be issued before problems were encountered. 



The Test provided a very accurate prediction 
of future performance. Memorizing tlie 28 Russian 
Pas to the initial criterion mentioned above, the 
following correlations were obtained: 



Test Score (Max 24) 



Ave rage 
Merorizing Time (Mi r. s) 



2 2 
18 
16 
12 



25 
3 5 
40 
55 



Scores lielov 12 we re advised to split r^emoriring 
sessions int.; two or more parts. This enabled 
them to achieve the criterion without having their 
performance impaired by fatigue. 

It will be evident from the figures above 
that *ILAT V makes it possible to estinate the total 
time an individual will require to achieve a 
professionally useful standard of performance. If 
a student does not have the requisite amount of time 
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to spare in Ii i s schedule, he can be advised not to 
corncnce the langunpe study. Pnilure in this field 
is often the product of bad planning and tliis-can 
no«- be avoided. 

Ihe average score for natural science under- 
oradiiatcs was 16» For others it was IS. Tlie 
difference is pr^obably due to the fact tliat nost 
of t iu- scientists \.erc men and a larpe proportion 
of tlic others were votren. Carroll and Sapon also 
notcU a narked difference between the performance 
of t:i'/ sexes. The firures in the present case are, 
hnut-ver, of linited interest because the non- 
scientists had nore lan;:ua.'re learning experience 
and this nay have had ?one influence on the-J^cores. 
In fe.'ieral the tcr.t i.is well dcsii^ned to nininize 
such factors. It bcrii^s witii data nrosented as if 
for a recall task, uhich turns out to he a inultiple 
chf^ico rroblerr, and this is likely to r'.duce the 
nJinrrit'cs of previously Irarnt tochniuiscs. 
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APPENDIX 



Course Mark Analysis of Variance 
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